You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 17:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Jul 20, 2026 12:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26014991043,92cyclictest23437-21apt-get08:26:521
260239910185,11cyclictest26000-21cyclictest07:36:402
260239910185,11cyclictest26000-21cyclictest07:36:402
260149910146,48cyclictest10277-21munin-plugin-st07:46:401
2601499986,81cyclictest25946-21apt-get10:31:541
26014999728,11cyclictest0-21swapper/108:46:501
26014999635,53cyclictest0-21swapper/107:27:071
26014999635,53cyclictest0-21swapper/107:27:071
26014999627,11cyclictest0-21swapper/111:47:101
26009999574,15cyclictest20010-21gpgv08:16:420
26023999473,11cyclictest0-21swapper/208:07:002
26023999472,11cyclictest0-21swapper/208:57:152
26023999433,32cyclictest39-21ksoftirqd/207:10:342
26014999470,18cyclictest991-21diskmemload10:21:591
2601499943,38cyclictest8439-21ps10:02:081
2601499943,37cyclictest24612-21mailstats11:22:121
2601499943,37cyclictest24612-21mailstats11:22:121
26014999420,12cyclictest0-21swapper/111:57:001
26009999472,11cyclictest0-21swapper/009:15:520
26023999375,13cyclictest26000-21cyclictest11:31:372
2601499934,33cyclictest18475-21cstates12:06:541
26014999341,44cyclictest11761-21latency_hist11:01:411
26014999341,44cyclictest11761-21latency_hist11:01:411
2601499933,37cyclictest13593-21latency_hist07:56:401
26014999332,55cyclictest0-21swapper/107:37:001
26009999376,11cyclictest26814-21ssh11:26:410
26023999271,12cyclictest0-21swapper/207:42:142
2601499926,74cyclictest27509-21apt-get12:21:531
26014999265,17cyclictest0-21swapper/110:15:171
2601499923,66cyclictest29175-21apt-get10:36:591
2601499923,40cyclictest5708-21latency_hist10:51:391
2601499923,37cyclictest2459-21/usr/sbin/munin07:22:121
26023999169,12cyclictest0-21swapper/208:51:402
26023999169,12cyclictest0-21swapper/207:31:412
26023999168,13cyclictest0-21swapper/208:21:562
26023999148,12cyclictest181rcu_preempt09:27:142
2602399913,62cyclictest29175-21apt-get10:37:012
2601499914,35cyclictest6965-21cron09:06:371
2601499914,34cyclictest25623-21if_eth009:37:011
2601499914,34cyclictest17903-21/usr/sbin/munin11:11:481
2601499913,40cyclictest2084-21fschecks_time09:52:001
2601499913,37cyclictest25744-21latency_hist10:31:421
2601499913,36cyclictest32174-21cstates08:51:561
26014999132,53cyclictest0-21swapper/108:06:381
26014999128,7cyclictest0-21swapper/108:22:151
26009999171,12cyclictest0-21swapper/009:42:160
26023999074,10cyclictest26783-21ntp_kernel_pll_10:32:512
26023999073,10cyclictest0-21swapper/207:52:102
2601499903,40cyclictest13238-21sh11:02:531
2601499903,35cyclictest30339-21sort07:07:001
26014999029,52cyclictest0-21swapper/108:07:151
26014999029,10cyclictest0-21swapper/112:21:411
26014999029,10cyclictest0-21swapper/112:21:411
2601499902,36cyclictest19494-21timerwakeupswit08:12:151
2601499902,34cyclictest16728-21ssh10:16:491
2601499902,34cyclictest16728-21ssh10:16:491
2601499902,33cyclictest18170-21latency_hist12:06:401
2601499902,33cyclictest18170-21latency_hist12:06:401
2601499902,31cyclictest10288-21/usr/sbin/munin09:12:041
26023998969,10cyclictest0-21swapper/207:03:002
2601499894,28cyclictest4988-21fschecks_count09:56:591
2601499894,28cyclictest4988-21fschecks_count09:56:591
2601499893,36cyclictest8753-21latency_hist10:56:401
2601499892,40cyclictest23760-21latency_hist11:21:411
2601499892,37cyclictest31945-21fschecks_time07:11:581
2601499892,30cyclictest28096-21grep08:37:141
2601499892,30cyclictest28096-21grep08:37:141
26009998970,11cyclictest0-21swapper/007:42:010
26009998910,40cyclictest15-21ksoftirqd/008:26:380
2601499882,33cyclictest26684-21grep06:57:051
2601499882,33cyclictest26684-21grep06:57:051
26009998866,12cyclictest0-21swapper/009:37:000
247742888,56sleep124811-21fschecks_time06:52:061
26014998735,44cyclictest0-21swapper/111:52:051
26014998735,44cyclictest0-21swapper/111:52:051
2601499873,32cyclictest16549-21fschecks_time09:22:011
2601499873,31cyclictest352-21runrttasks08:32:151
2601499873,31cyclictest352-21runrttasks08:32:151
2601499873,31cyclictest20463-21/usr/sbin/munin08:16:491
2601499873,30cyclictest20172-21timerandwakeup09:27:141
26014998725,13cyclictest0-21swapper/108:57:041
2601499872,35cyclictest25045-21ssh09:36:401
2601499872,34cyclictest13432-21munin-run10:11:381
2601499872,33cyclictest29483-21head07:03:051
2601499872,32cyclictest1383-21grep07:17:061
26014998718,20cyclictest15557-21sh11:07:091
26014998718,20cyclictest15557-21sh11:07:091
26009998768,13cyclictest20976-21runrttasks11:16:560
2601499865,47cyclictest32390-21apt-get10:42:191
2601499864,35cyclictest27640-21sendmail_mailtr11:27:111
2601499863,34cyclictest6349-21awk07:32:111
2601499863,34cyclictest6349-21awk07:32:111
2601499863,30cyclictest10787-21fschecks_time07:47:001
2601499862,30cyclictest29016-21fschecks_time08:42:001
2601499862,29cyclictest28486-21fschecks_count09:41:591
26009998666,11cyclictest0-21swapper/007:21:400
26009998660,18cyclictest1170-21latency_hist08:56:410
26009998651,26cyclictest4315-21cut07:26:590
26009998651,26cyclictest4315-21cut07:26:590
2462828618,36sleep20-21swapper/206:51:592
26023998567,11cyclictest25829-21ssh12:18:262
26023998567,11cyclictest25829-21ssh12:18:262
26023998567,10cyclictest0-21swapper/207:21:412
26023998562,17cyclictest30242-21latency_hist08:46:402
2601499853,33cyclictest3932-21/usr/sbin/munin11:42:071
2601499853,32cyclictest7784-21grep09:07:051
2601499853,32cyclictest7784-21grep09:07:051
2601499853,31cyclictest31224-21/usr/sbin/munin09:47:111
2601499852,36cyclictest1263-21awk11:37:131
26009998566,10cyclictest0-21swapper/012:17:000
26009998566,10cyclictest0-21swapper/012:17:000
26009998532,24cyclictest15-21ksoftirqd/007:26:340
26009998531,16cyclictest181rcu_preempt10:30:130
26023998462,13cyclictest0-21swapper/209:26:402
26023998462,11cyclictest0-21swapper/209:17:112
2602399843,47cyclictest39-21ksoftirqd/207:26:592
2602399842,49cyclictest39-21ksoftirqd/209:10:192
2602399842,49cyclictest39-21ksoftirqd/209:10:192
2601499844,35cyclictest352-21runrttasks11:31:481
2601499843,34cyclictest13808-21cpu07:56:551
2601499843,34cyclictest13808-21cpu07:56:551
26014998433,43cyclictest0-21swapper/109:20:001
26009998466,9cyclictest0-21swapper/010:21:400
26009998466,9cyclictest0-21swapper/010:21:400
26009998463,14cyclictest298-21in:imuxsock09:56:380
26009998444,31cyclictest0-21swapper/009:57:100
26009998444,31cyclictest0-21swapper/009:57:100
2600999842,49cyclictest15-21ksoftirqd/010:39:560
26023998366,10cyclictest0-21swapper/211:32:022
26023998338,10cyclictest0-21swapper/206:56:482
26023998338,10cyclictest0-21swapper/206:56:482
2600999837,45cyclictest15-21ksoftirqd/011:06:540
2600999837,45cyclictest15-21ksoftirqd/011:06:540
2600999834,45cyclictest15-21ksoftirqd/010:53:440
2600999832,45cyclictest15-21ksoftirqd/011:55:390
2600999832,45cyclictest15-21ksoftirqd/011:55:390
26023998266,8cyclictest0-21swapper/208:52:012
26023998263,9cyclictest0-21swapper/209:51:392
26023998260,13cyclictest0-21swapper/209:32:152
26023998260,12cyclictest0-21swapper/208:31:402
26009998263,7cyclictest181rcu_preempt12:22:280
26009998261,11cyclictest0-21swapper/012:06:380
26009998261,11cyclictest0-21swapper/012:06:380
26009998251,13cyclictest181rcu_preempt11:56:540
26023998164,9cyclictest0-21swapper/209:37:152
26023998160,9cyclictest181rcu_preempt11:02:132
26023998155,15cyclictest0-21swapper/207:12:102
26023998151,20cyclictest0-21swapper/210:02:122
26023998133,22cyclictest39-21ksoftirqd/211:58:152
2602399812,52cyclictest39-21ksoftirqd/211:08:262
2602399812,52cyclictest39-21ksoftirqd/211:08:262
2601499814,26cyclictest21310-21/usr/sbin/munin12:12:021
2600999813,42cyclictest15-21ksoftirqd/008:11:190
26009998120,27cyclictest15-21ksoftirqd/006:59:020
26009998120,27cyclictest15-21ksoftirqd/006:59:020
26023998062,11cyclictest0-21swapper/212:02:142
26023998062,11cyclictest0-21swapper/212:02:142
26023998059,15cyclictest25686-21apt-key10:31:412
26023998056,16cyclictest1-21systemd10:46:392
26023998049,22cyclictest6027-21modprobe11:46:372
26023998033,12cyclictest181rcu_preempt09:01:572
26009998052,21cyclictest0-21swapper/009:01:500
26009998037,13cyclictest181rcu_preempt11:48:180
2600999803,42cyclictest15-21ksoftirqd/011:45:350
2600999802,41cyclictest15-21ksoftirqd/010:44:080
26009998018,33cyclictest15-21ksoftirqd/008:04:380
26023997958,14cyclictest17460-21sshd07:22:072
26009997958,7cyclictest181rcu_preempt12:08:020
26009997951,20cyclictest0-21swapper/009:29:520
26009997949,11cyclictest181rcu_preempt07:14:330
26009997935,13cyclictest181rcu_preempt10:23:550
2600999793,46cyclictest15-21ksoftirqd/011:37:160
2600999793,37cyclictest15-21ksoftirqd/009:01:410
26009997933,16cyclictest0-21swapper/010:07:060
26009997932,18cyclictest181rcu_preempt10:02:450
2603099782,46cyclictest23977-21taskset08:26:533
26023997834,12cyclictest181rcu_preempt08:41:382
26023997834,12cyclictest181rcu_preempt08:41:382
26009997860,12cyclictest26000-21cyclictest12:15:300
26009997857,12cyclictest0-21swapper/010:32:470
26009997817,30cyclictest15-21ksoftirqd/007:51:110
26023997753,14cyclictest0-21swapper/210:51:402
26009997735,20cyclictest181rcu_preempt10:12:540
2600999773,37cyclictest15-21ksoftirqd/009:19:470
26009997727,16cyclictest181rcu_preempt07:08:130
26023997659,9cyclictest0-21swapper/210:52:072
26023997653,16cyclictest26000-21cyclictest11:13:162
2602399763,44cyclictest39-21ksoftirqd/207:37:072
2602399762,47cyclictest39-21ksoftirqd/207:48:422
26009997653,14cyclictest0-21swapper/008:51:410
26009997647,9cyclictest181rcu_preempt07:35:450
26009997647,9cyclictest181rcu_preempt07:35:450
2600999763,35cyclictest15-21ksoftirqd/008:26:510
26023997558,8cyclictest0-21swapper/212:12:142
26023997554,11cyclictest0-21swapper/210:16:402
26023997536,12cyclictest181rcu_preempt11:22:402
26023997536,12cyclictest181rcu_preempt11:22:402
2600999755,44cyclictest15-21ksoftirqd/008:34:190
2600999755,44cyclictest15-21ksoftirqd/008:34:190
2600999754,45cyclictest15-21ksoftirqd/009:11:380
2600999754,45cyclictest15-21ksoftirqd/009:11:380
2600999754,40cyclictest15-21ksoftirqd/009:32:060
26009997525,26cyclictest15-21ksoftirqd/008:39:300
26009997525,26cyclictest15-21ksoftirqd/008:39:300
2602399747,42cyclictest39-21ksoftirqd/212:10:022
2602399747,32cyclictest39-21ksoftirqd/210:19:442
2602399747,32cyclictest39-21ksoftirqd/210:19:442
26023997443,11cyclictest181rcu_preempt11:55:542
26023997443,11cyclictest181rcu_preempt11:55:542
26023997434,11cyclictest181rcu_preempt11:39:522
26023997425,12cyclictest181rcu_preempt10:58:432
26023997425,12cyclictest181rcu_preempt10:58:432
26009997457,10cyclictest0-21swapper/007:51:560
26009997455,9cyclictest0-21swapper/008:46:410
2600999744,48cyclictest27536-21apt-get07:01:520
2600999744,45cyclictest15-21ksoftirqd/010:57:390
2600999744,45cyclictest15-21ksoftirqd/010:57:390
2600999742,45cyclictest15-21ksoftirqd/007:57:580
2600999742,45cyclictest15-21ksoftirqd/007:57:580
26009997416,36cyclictest15-21ksoftirqd/011:22:050
26009997416,36cyclictest15-21ksoftirqd/011:22:050
26009997410,38cyclictest15-21ksoftirqd/010:46:560
26023997352,12cyclictest0-21swapper/208:16:392
2600999739,31cyclictest17281-21grep09:22:160
26009997343,8cyclictest181rcu_preempt11:13:320
2600999733,40cyclictest15-21ksoftirqd/011:31:550
2600999733,29cyclictest15-21ksoftirqd/007:36:580
2602399728,26cyclictest39-21ksoftirqd/209:41:442
26023997255,12cyclictest0-21swapper/208:18:102
26023997253,10cyclictest0-21swapper/210:23:102
26023997250,12cyclictest0-21swapper/210:10:422
26023997234,7cyclictest181rcu_preempt08:01:412
2600999725,39cyclictest15-21ksoftirqd/011:02:010
2600999725,39cyclictest15-21ksoftirqd/009:47:130
26023997154,11cyclictest2456-21proc_pri09:52:102
26023997129,9cyclictest181rcu_preempt12:24:292
26023997048,15cyclictest25820-21cut08:32:022
26023997048,15cyclictest25820-21cut08:32:022
26023997045,15cyclictest991-21diskmemload09:11:592
26023997030,13cyclictest181rcu_preempt09:59:242
26023997030,13cyclictest181rcu_preempt09:59:242
26023997028,19cyclictest39-21ksoftirqd/211:47:042
2600999705,36cyclictest15-21ksoftirqd/008:11:500
26023996953,11cyclictest2374-21systemd-journal07:59:522
26023996953,11cyclictest2374-21systemd-journal07:59:522
26023996943,7cyclictest181rcu_preempt11:17:432
250042639,31sleep025037-21ntpq06:52:130
251332609,26sleep30-21swapper/306:52:213
2603099564,37cyclictest6959-21taskset09:06:363
2603099553,40cyclictest372-21taskset09:48:363
2603099543,34cyclictest16606-21taskset10:16:413
2603099542,36cyclictest13840-21taskset07:56:553
2603099542,36cyclictest13840-21taskset07:56:553
2603099533,39cyclictest19956-21chrt10:22:003
2603099533,38cyclictest5971-21chrt07:32:003
2603099533,38cyclictest5971-21chrt07:32:003
2603099533,37cyclictest17628-21chrt10:17:133
2603099533,37cyclictest17628-21chrt10:17:133
2603099533,36cyclictest32378-21chrt11:36:133
2603099533,34cyclictest29123-21chrt10:36:433
2603099524,34cyclictest17710-21taskset08:07:123
2603099524,33cyclictest7097-21chrt11:47:113
2603099523,36cyclictest25549-21taskset10:31:403
2603099523,36cyclictest11677-21taskset11:01:403
2603099523,36cyclictest11677-21taskset11:01:403
2603099523,30cyclictest27990-21chrt07:01:423
2603099513,36cyclictest6814-21taskset09:59:553
2603099513,36cyclictest6814-21taskset09:59:553
2603099513,35cyclictest21091-21chrt12:11:393
2603099513,33cyclictest22630-21chrt09:32:023
2603099512,37cyclictest27247-21chrt11:27:013
2603099512,36cyclictest19118-21taskset08:12:063
2603099512,35cyclictest11355-21taskset09:12:163
2603099504,35cyclictest3023-21taskset08:58:583
2603099503,37cyclictest29517-21chrt09:42:443
2603099503,35cyclictest28720-21chrt12:22:503
2603099503,34cyclictest7963-21taskset10:54:513
2603099503,34cyclictest3531-21chrt11:41:563
2603099503,34cyclictest27155-21chrt12:21:403
2603099503,34cyclictest27155-21chrt12:21:403
2603099503,33cyclictest14252-21chrt09:17:143
2603099503,30cyclictest19230-21chrt09:26:433
2603099502,35cyclictest16632-21chrt11:08:573
2603099502,35cyclictest16632-21chrt11:08:573
2603099493,34cyclictest23809-21chrt12:16:093
2603099493,33cyclictest8232-21chrt07:37:153
2603099493,33cyclictest30925-21chrt08:47:063
2603099493,31cyclictest9211-21taskset07:42:013
2603099492,35cyclictest25736-21chrt08:31:593
2603099492,35cyclictest25736-21chrt08:31:593
2603099483,32cyclictest32479-21chrt07:12:113
2603099483,31cyclictest12629-21chrt11:57:003
2603099483,30cyclictest16546-21chrt09:22:013
2603099482,37cyclictest13118-21taskset11:02:333
2603099482,33cyclictest3164-21chrt11:41:403
2603099482,32cyclictest17284-21chrt12:04:383
2603099482,32cyclictest17284-21chrt12:04:383
2603099482,32cyclictest13553-21chrt07:56:393
2603099482,30cyclictest20502-21taskset08:17:013
2603099473,33cyclictest8389-21chrt09:07:173
2603099473,33cyclictest8389-21chrt09:07:173
2603099473,32cyclictest27551-21taskset07:01:383
2603099473,30cyclictest20976-21taskset11:16:563
2603099472,31cyclictest24774-21chrt11:22:153
2603099472,31cyclictest24774-21chrt11:22:153
2603099462,35cyclictest25936-21chrt10:31:483
2603099462,32cyclictest19027-21chrt11:12:183
2603099453,33cyclictest32711-21chrt10:42:233
2603099453,31cyclictest2247-21taskset07:21:393
2603099452,30cyclictest25882-21chrt09:37:093
2603099447,25cyclictest0-21swapper/307:02:223
2603099443,32cyclictest1968-21chrt09:51:563
2603099433,29cyclictest12042-21taskset07:51:413
2603099424,27cyclictest7997-21taskset10:01:573
2603099422,25cyclictest9304-21chrt11:51:413
2603099422,25cyclictest9304-21chrt11:51:413
2603099413,27cyclictest28773-21taskset08:41:483
2603099412,29cyclictest11563-21taskset10:07:123
2603099403,29cyclictest4533-21taskset10:48:353
2603099403,28cyclictest2625-21taskset07:21:553
2603099403,26cyclictest4125-21chrt07:26:543
2603099403,26cyclictest4125-21chrt07:26:543
2603099402,27cyclictest30115-21chrt07:06:493
2603099393,27cyclictest28263-21taskset08:38:193
2603099393,27cyclictest28263-21taskset08:38:193
2603099393,27cyclictest23106-21taskset08:22:323
2603099393,27cyclictest16706-21taskset08:06:233
2603099393,26cyclictest894-21taskset08:55:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional