You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 01:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Jul 08, 2026 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
183321349,95sleep01265-21/usr/sbin/munin06:53:070
1824211211,72sleep21830-21/usr/sbin/munin06:53:052
248821026,16sleep10-21swapper/106:55:581
275399912,65cyclictest32668-21sh10:28:171
2746999155,30cyclictest32346-21switchtime09:33:030
2746999155,30cyclictest32346-21switchtime09:33:030
274699912,71cyclictest32427-21apt-get10:28:080
2753999065,7cyclictest181rcu_preempt09:46:241
2753999065,7cyclictest181rcu_preempt09:46:241
2753999043,22cyclictest4895-21sensors_temp07:02:401
2746999020,25cyclictest0-21swapper/011:23:500
2753998847,18cyclictest31-21ksoftirqd/109:06:541
2753998847,18cyclictest31-21ksoftirqd/109:06:541
2746998867,14cyclictest1263-21ssh12:17:510
2746998867,14cyclictest1263-21ssh12:17:510
2755998671,9cyclictest6669-21modprobe08:47:272
2755998671,9cyclictest6669-21modprobe08:47:272
2753998619,40cyclictest31-21ksoftirqd/109:49:531
2746998617,27cyclictest0-21swapper/010:05:230
129692850,9sleep20-21swapper/207:27:302
2753998448,8cyclictest181rcu_preempt11:45:351
2753998448,8cyclictest181rcu_preempt11:45:351
2746998412,31cyclictest5516-21taskset07:03:000
275399838,43cyclictest21155-21apt-get11:02:451
275399838,43cyclictest21155-21apt-get11:02:451
2753998358,19cyclictest28592-21cpuspeed09:27:441
2753998316,38cyclictest31-21ksoftirqd/111:02:061
2755998263,12cyclictest23427-21sh10:12:492
2755998263,12cyclictest23427-21sh10:12:492
2753998255,17cyclictest9911-21diskmemload09:17:301
274699822,66cyclictest19571-21sh09:12:570
2755998163,11cyclictest2059-21ssh11:24:182
2753998115,40cyclictest31-21ksoftirqd/110:14:381
2753998115,40cyclictest31-21ksoftirqd/110:14:381
2746998162,13cyclictest22775-21apt-key07:57:310
2746998156,18cyclictest29472-21apt-key08:17:310
2746998142,14cyclictest15-21ksoftirqd/011:47:490
2755998065,11cyclictest2738-21cyclictest07:07:272
2755998063,11cyclictest6156-21latency_hist11:32:292
2755998063,11cyclictest6156-21latency_hist11:32:292
2755998059,14cyclictest8589-21sh11:36:012
2755998017,25cyclictest5941-21/usr/sbin/munin10:37:502
2755998017,25cyclictest5941-21/usr/sbin/munin10:37:502
2753998052,8cyclictest181rcu_preempt08:18:011
2753998016,40cyclictest31-21ksoftirqd/106:59:541
2746998048,23cyclictest7954-21modprobe07:12:270
2755997954,19cyclictest24610-21kworker/u19:1+nfsiod09:46:562
2755997954,19cyclictest24610-21kworker/u19:1+nfsiod09:46:562
2755997925,22cyclictest16203-21runrttasks09:07:352
2755997922,25cyclictest1471-21unixbench_multi10:29:272
2755997922,22cyclictest0-21swapper/211:39:072
2755997918,26cyclictest0-21swapper/208:42:282
2753997962,11cyclictest9911-21diskmemload12:01:191
2753997958,7cyclictest181rcu_preempt12:24:111
2753997957,16cyclictest655-21sh11:22:421
2753997939,20cyclictest2738-21cyclictest12:18:101
2753997939,20cyclictest2738-21cyclictest12:18:101
2753997910,34cyclictest31-21ksoftirqd/108:23:221
274699799,28cyclictest26087-21rm12:05:140
2746997964,9cyclictest24323-21ssh11:07:480
2746997957,14cyclictest26222-21sh10:17:420
2746997918,29cyclictest16107-21latency_hist09:07:300
2746997918,29cyclictest16107-21latency_hist09:07:300
2746997911,29cyclictest9662-21rm09:49:330
2746997910,28cyclictest17213-21ssh10:56:200
11952796,15sleep30-21swapper/306:52:333
7632780,9sleep30-21swapper/312:17:363
7632780,9sleep30-21swapper/312:17:363
2755997847,24cyclictest0-21swapper/209:02:512
2755997847,24cyclictest0-21swapper/209:02:512
2755997831,35cyclictest28504-21apt-get09:27:382
2755997820,28cyclictest32123-21runrttasks11:20:572
2755997816,24cyclictest0-21swapper/209:56:332
2753997859,7cyclictest181rcu_preempt11:54:471
2753997857,8cyclictest181rcu_preempt09:25:421
2753997857,8cyclictest181rcu_preempt09:25:421
2753997847,7cyclictest181rcu_preempt11:11:011
2753997839,11cyclictest12952-21sh09:54:461
2753997834,12cyclictest0-21swapper/110:43:161
2755997728,20cyclictest17317-21sh11:50:302
2755997725,24cyclictest9911-21diskmemload09:26:322
2755997725,24cyclictest9911-21diskmemload09:26:322
2755997725,23cyclictest1269-21kworker/u19:2+rpciod09:48:422
2753997753,11cyclictest181rcu_preempt07:27:081
2753997742,8cyclictest181rcu_preempt07:48:011
2753997737,8cyclictest181rcu_preempt10:22:351
2753997737,8cyclictest181rcu_preempt10:22:351
2753997732,10cyclictest0-21swapper/109:33:011
2753997732,10cyclictest0-21swapper/109:33:011
274699779,28cyclictest22307-21sh10:10:480
2746997711,26cyclictest19397-21rm10:59:040
2746997711,24cyclictest11345-21ssh09:52:540
2755997657,14cyclictest32534-21kworker/u17:3+events_unbound08:37:272
2755997656,14cyclictest14455-21latency_hist07:32:292
2755997656,14cyclictest14455-21latency_hist07:32:292
2755997640,23cyclictest2943-21apt-get06:57:432
2755997626,19cyclictest9911-21diskmemload12:24:502
2755997613,29cyclictest0-21swapper/210:42:462
2753997647,7cyclictest181rcu_preempt08:43:011
2753997647,7cyclictest181rcu_preempt08:43:011
2753997641,20cyclictest9984-21df_inode07:17:461
2753997621,22cyclictest31-21ksoftirqd/111:16:141
2753997621,22cyclictest31-21ksoftirqd/111:16:141
2753997619,32cyclictest31-21ksoftirqd/109:00:251
2753997615,37cyclictest31-21ksoftirqd/111:28:271
2753997615,37cyclictest31-21ksoftirqd/111:28:271
274699769,26cyclictest6994-21rm09:45:320
274699769,26cyclictest6994-21rm09:45:320
274699769,26cyclictest14505-21ssh10:52:080
274699768,28cyclictest27556-21ssh09:25:470
274699768,28cyclictest27556-21ssh09:25:470
274699767,28cyclictest16165-21rm10:00:240
274699767,28cyclictest16165-21rm10:00:240
2746997623,36cyclictest2007-21sort09:37:360
2755997559,10cyclictest765-21mandb12:17:522
2755997559,10cyclictest765-21mandb12:17:522
2755997559,10cyclictest2738-21cyclictest09:17:302
2755997558,10cyclictest31582-21cpuspeed09:32:442
2755997558,10cyclictest31582-21cpuspeed09:32:442
2755997528,15cyclictest20652-21taskset07:49:472
2755997527,25cyclictest13763-21seq10:50:022
2755997518,27cyclictest30593-21ssh12:12:512
275399757,34cyclictest31-21ksoftirqd/110:53:011
275399756,41cyclictest31-21ksoftirqd/107:11:531
2753997552,8cyclictest181rcu_preempt12:03:041
2753997534,12cyclictest181rcu_preempt07:42:491
274699759,27cyclictest5503-21rm11:30:360
274699759,27cyclictest5503-21rm11:30:360
274699758,27cyclictest18385-21rm09:11:010
2746997531,37cyclictest4218-21ntp_kernel_pll_08:37:560
2746997511,24cyclictest5872-21sh12:24:440
2755997433,30cyclictest0-21swapper/212:07:382
2755997433,30cyclictest0-21swapper/212:07:382
2755997427,22cyclictest9814-21seq08:56:362
2755997427,19cyclictest9911-21diskmemload10:55:232
2755997422,25cyclictest352-21runrttasks08:24:362
2755997422,25cyclictest352-21runrttasks08:15:322
2755997415,23cyclictest352-21runrttasks07:32:582
2755997415,23cyclictest352-21runrttasks07:32:582
2753997445,7cyclictest181rcu_preempt12:13:451
2753997435,12cyclictest10694-21ssh11:38:321
2753997428,12cyclictest0-21swapper/108:32:281
274699748,26cyclictest24290-21ssh09:20:000
2746997412,24cyclictest31500-21rm11:19:100
2746997412,23cyclictest32126-21sh12:15:290
212442740,14sleep0161ktimers/011:57:340
2755997358,10cyclictest19903-21sendmail10:07:262
2755997342,24cyclictest0-21swapper/207:17:282
2755997338,30cyclictest0-21swapper/211:42:392
2755997338,30cyclictest0-21swapper/211:42:392
2755997334,30cyclictest29191-21apt-get08:17:412
2755997333,28cyclictest20964-21latency_hist11:02:302
2755997327,24cyclictest352-21runrttasks07:07:392
2755997325,18cyclictest9911-21diskmemload09:18:542
2755997324,19cyclictest352-21runrttasks09:58:142
2755997324,19cyclictest352-21runrttasks09:58:142
2755997323,17cyclictest9911-21diskmemload10:09:372
275399739,30cyclictest31-21ksoftirqd/107:37:331
275399739,30cyclictest31-21ksoftirqd/107:37:331
2753997356,12cyclictest16663-21kernelversion09:07:521
2753997334,9cyclictest181rcu_preempt10:10:371
275399733,38cyclictest16902-21latency_hist10:02:291
275399733,38cyclictest16902-21latency_hist10:02:291
2753997319,32cyclictest31-21ksoftirqd/110:50:021
274699737,28cyclictest29076-21rm11:15:400
274699737,28cyclictest29076-21rm11:15:400
274699737,25cyclictest14354-21ssh11:45:340
274699737,25cyclictest14354-21ssh11:45:340
2746997351,10cyclictest2738-21cyclictest10:35:270
2746997327,26cyclictest27380-21kworker/u17:2+xprtiod10:26:070
2746997327,26cyclictest27380-21kworker/u17:2+xprtiod10:26:070
2746997314,20cyclictest8410-21sh11:35:280
2755997236,27cyclictest21148-21chrt07:52:422
2755997234,28cyclictest27598-21awk08:12:272
2755997231,30cyclictest10591-21/usr/sbin/munin08:57:482
2755997228,35cyclictest24597-21/usr/sbin/munin08:02:372
2755997227,22cyclictest19130-21seq07:46:202
2755997223,22cyclictest0-21swapper/210:18:562
2753997251,8cyclictest181rcu_preempt07:35:511
2753997251,8cyclictest181rcu_preempt07:35:511
2753997248,8cyclictest181rcu_preempt08:37:381
2753997229,20cyclictest0-21swapper/109:39:111
2753997213,21cyclictest301ktimers/108:12:451
274699728,24cyclictest30791-21rm09:31:250
2746997242,21cyclictest8285-21modprobe08:52:260
2746997210,24cyclictest11040-21ssh10:45:520
2755997148,14cyclictest2968-21apt-get10:32:412
2755997140,23cyclictest0-21swapper/207:38:052
2755997140,23cyclictest0-21swapper/207:38:052
2755997132,29cyclictest20613-21apt-get11:02:422
2755997132,29cyclictest20613-21apt-get11:02:422
2755997125,17cyclictest25682-21sh12:04:062
2755997122,21cyclictest0-21swapper/209:38:592
2753997153,7cyclictest181rcu_preempt11:48:271
2753997150,8cyclictest181rcu_preempt07:31:491
2753997150,8cyclictest181rcu_preempt07:31:491
2753997148,8cyclictest181rcu_preempt10:38:011
2753997148,8cyclictest181rcu_preempt10:38:011
2753997138,9cyclictest181rcu_preempt10:34:131
2753997135,10cyclictest181rcu_preempt12:09:481
2753997135,10cyclictest181rcu_preempt12:09:481
2753997129,32cyclictest22736-21sh09:17:501
2753997123,12cyclictest0-21swapper/111:33:181
274699718,26cyclictest8474-21rm10:42:020
274699718,26cyclictest8474-21rm10:42:020
274699716,26cyclictest11822-21ssh11:41:450
2746997153,12cyclictest2374-21systemd-journal12:12:270
2746997153,12cyclictest2374-21systemd-journal12:12:270
2746997152,13cyclictest9309-21threads08:53:020
2746997150,16cyclictest544-21cron10:17:270
2746997150,16cyclictest544-21cron10:17:270
2746997121,33cyclictest0-21swapper/007:33:010
2746997121,33cyclictest0-21swapper/007:33:010
2746997112,25cyclictest18359-21ssh11:52:420
2755997024,24cyclictest352-21runrttasks11:16:432
2755997024,24cyclictest352-21runrttasks11:16:432
2753997044,9cyclictest181rcu_preempt07:59:491
275399702,38cyclictest31-21ksoftirqd/110:05:041
2753997010,36cyclictest31-21ksoftirqd/111:20:361
274699709,22cyclictest22751-21ssh11:04:340
274699709,22cyclictest22751-21ssh11:04:340
2746997056,8cyclictest0-21swapper/007:58:240
274699705,24cyclictest19199-21taskset07:47:110
2746997051,13cyclictest300-21rs:main0
2746997051,13cyclictest300-21rs:main0
2755996938,19cyclictest29177-21apt-get10:22:362
2755996938,19cyclictest29177-21apt-get10:22:362
2755996930,30cyclictest24482-21/usr/sbin/munin11:07:502
2755996912,26cyclictest0-21swapper/211:55:292
2753996949,14cyclictest3636-21latency_hist08:37:301
2753996927,17cyclictest0-21swapper/108:57:281
2753996927,12cyclictest0-21swapper/107:56:581
2753996925,9cyclictest181rcu_preempt08:10:141
2753996917,32cyclictest31-21ksoftirqd/110:20:301
2755996835,25cyclictest7416-21awk08:47:552
275399683,41cyclictest8282-21modprobe08:52:261
274699686,22cyclictest9455-21taskset07:16:120
2746996847,13cyclictest3609-21runrttasks08:37:300
2746996811,25cyclictest447-21seq08:27:320
2755996732,24cyclictest22987-21apt07:57:382
2755996732,24cyclictest21269-21apt-get11:57:412
2755996728,29cyclictest1820-21latency_hist08:32:302
2753996732,7cyclictest181rcu_preempt07:14:061
2753996715,30cyclictest31-21ksoftirqd/108:03:341
9812660,6sleep30-21swapper/309:35:283
9812660,6sleep30-21swapper/309:35:283
2755996637,19cyclictest9835-21apt-get07:17:392
2746996644,15cyclictest31780-21ntp_kernel_err08:22:550
2746996534,23cyclictest0-21swapper/008:22:290
2746996441,14cyclictest11668-21fschecks_time07:22:500
2746996437,19cyclictest19929-21latency07:47:520
2746996411,17cyclictest4174-21seq07:01:150
2766996240,14cyclictest451rcuc/306:59:543
2746996245,11cyclictest11161-21cron07:22:270
274699624,17cyclictest13291-21fschecks_count07:27:500
274699624,17cyclictest13291-21fschecks_count07:27:500
274699617,22cyclictest6374-21seq08:44:040
274699617,22cyclictest6374-21seq08:44:040
2746996110,43cyclictest12381-21ssh09:01:020
2746996044,10cyclictest26302-21cpuspeed08:07:430
274699604,29cyclictest25242-21mailstats08:03:000
2766995744,9cyclictest451rcuc/311:38:343
2766995744,9cyclictest451rcuc/311:12:013
228732570,9sleep30-21swapper/309:17:523
2766995643,9cyclictest451rcuc/307:26:513
2766995543,8cyclictest451rcuc/311:31:153
2766995543,8cyclictest451rcuc/311:31:153
2766995543,8cyclictest451rcuc/307:15:463
2766995441,9cyclictest451rcuc/310:56:123
2766995440,9cyclictest451rcuc/310:14:383
2766995440,9cyclictest451rcuc/310:14:383
2766995434,12cyclictest451rcuc/309:31:073
2766995434,12cyclictest451rcuc/307:03:183
276699543,38cyclictest19837-21taskset11:00:193
2766995341,8cyclictest451rcuc/311:53:533
2766995340,9cyclictest451rcuc/308:25:333
276699523,34cyclictest4213-21taskset10:33:263
2766995138,9cyclictest451rcuc/312:08:193
2766995138,9cyclictest451rcuc/312:08:193
2766995138,9cyclictest451rcuc/309:43:313
2766995138,9cyclictest451rcuc/309:43:313
2766995138,8cyclictest451rcuc/307:43:573
2766995137,10cyclictest451rcuc/308:18:533
2766995134,12cyclictest451rcuc/311:24:293
276699512,34cyclictest29671-21taskset11:17:233
276699512,34cyclictest29671-21taskset11:17:233
276699512,33cyclictest8501-21taskset09:48:053
276699512,33cyclictest13213-21chrt09:02:393
276699512,33cyclictest13213-21chrt09:02:393
2766995037,8cyclictest451rcuc/308:29:023
2766995035,10cyclictest451rcuc/307:53:253
2766994937,8cyclictest451rcuc/308:43:153
2766994937,8cyclictest451rcuc/308:43:153
2766994936,9cyclictest451rcuc/311:45:583
2766994936,9cyclictest451rcuc/311:45:583
2766994936,9cyclictest451rcuc/307:58:353
276699492,31cyclictest21700-21taskset10:09:053
2766994836,8cyclictest451rcuc/310:51:443
2766994836,8cyclictest451rcuc/310:39:183
2766994836,8cyclictest451rcuc/310:39:183
2766994834,9cyclictest451rcuc/308:03:383
2766994834,10cyclictest451rcuc/307:39:313
2766994834,10cyclictest451rcuc/307:39:313
276699483,31cyclictest19012-21chrt09:12:283
2766994832,9cyclictest451rcuc/308:12:003
276699483,26cyclictest0-21swapper/310:28:103
276699482,33cyclictest21656-21chrt11:57:493
2766994734,9cyclictest451rcuc/307:48:353
2766994734,8cyclictest451rcuc/312:03:383
2766994734,8cyclictest451rcuc/311:19:173
2766994734,8cyclictest451rcuc/310:18:433
2766994733,9cyclictest451rcuc/311:07:153
2766994733,9cyclictest451rcuc/311:07:153
2766994733,9cyclictest451rcuc/307:20:073
2766994633,9cyclictest451rcuc/311:34:533
2766994632,9cyclictest451rcuc/307:34:373
2766994632,9cyclictest451rcuc/307:34:373
2766994632,10cyclictest451rcuc/312:15:263
2766994631,10cyclictest451rcuc/307:11:413
276699453,31cyclictest17547-21taskset10:02:493
2766994532,9cyclictest451rcuc/308:51:383
2766994532,8cyclictest451rcuc/308:34:043
2766994531,9cyclictest451rcuc/309:57:223
2766994531,9cyclictest451rcuc/308:56:203
276699443,27cyclictest7063-21chrt12:27:303
2766994431,9cyclictest451rcuc/309:16:033
2766994431,9cyclictest451rcuc/307:28:303
2766994431,9cyclictest451rcuc/307:28:303
276699442,28cyclictest2682-21chrt09:37:593
2766994330,9cyclictest451rcuc/310:42:323
276699423,29cyclictest10894-21taskset08:57:573
276699423,24cyclictest15615-21taskset09:58:513
276699423,24cyclictest15615-21taskset09:58:513
2766994228,9cyclictest451rcuc/311:48:543
2766994226,11cyclictest278252sleep308:12:373
276699422,26cyclictest29361-21taskset10:22:443
276699422,26cyclictest29361-21taskset10:22:443
276699413,28cyclictest25679-21taskset09:22:463
276699413,28cyclictest25679-21taskset09:22:463
276699403,27cyclictest3862-21taskset08:37:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional