You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 04:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Jul 29, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31665215713,94sleep231712-21cstates18:51:212
640991202,33cyclictest25505-21mailstats23:11:370
640991183,37cyclictest26604-21latency_hist22:21:060
6409911440,11cyclictest0-21swapper/019:26:390
640991142,43cyclictest1401-21cmp23:26:070
640991142,43cyclictest1401-21cmp23:26:070
2734821130,7sleep1618-21cyclictest20:16:251
6509911132,33cyclictest181rcu_preempt23:06:501
6409911139,19cyclictest31007-21grep22:26:400
640991103,39cyclictest28471-21/usr/sbin/munin00:11:290
640991103,39cyclictest28471-21/usr/sbin/munin00:11:290
3273721106,15sleep00-21swapper/018:53:270
6409910936,11cyclictest0-21swapper/023:41:060
6409910936,11cyclictest0-21swapper/023:41:060
6409910915,11cyclictest0-21swapper/021:26:390
640991073,31cyclictest16859-21ssh21:11:040
640991073,26cyclictest7986-21fschecks_time19:16:250
6409910642,14cyclictest31465-21/usr/sbin/munin00:16:240
6409910638,9cyclictest0-21swapper/023:55:260
6409910638,9cyclictest0-21swapper/023:55:260
640991063,38cyclictest11613-21modprobe21:56:030
640991063,38cyclictest11613-21modprobe21:56:030
640991063,29cyclictest14184-21/usr/sbin/munin23:46:360
6409910539,16cyclictest21823-21timerandwakeup22:11:370
640991053,39cyclictest2958-21/usr/sbin/munin20:42:440
640991053,37cyclictest20945-21tune2fs19:56:260
6409910533,12cyclictest0-21swapper/021:46:370
6409910516,34cyclictest28294-21sshd23:16:310
650991044,71cyclictest1854-21df23:26:471
650991044,71cyclictest1854-21df23:26:471
640991044,37cyclictest1671-21runrttasks21:38:300
6409910433,11cyclictest0-21swapper/000:10:080
6409910417,33cyclictest15558-21munin-run19:41:040
6409910417,33cyclictest15558-21munin-run19:41:040
6409910334,9cyclictest0-21swapper/020:57:030
640991033,44cyclictest11788-21/usr/sbin/munin21:01:380
640991033,38cyclictest12776-21fschecks_count22:51:270
640991033,37cyclictest6768-21grep19:11:370
640991033,33cyclictest5782-21latency_hist21:46:070
640991033,24cyclictest22188-21cpuspeed23:07:180
3263721035,14sleep10-21swapper/118:52:101
6409910237,12cyclictest352-21runrttasks18:56:410
6409910237,12cyclictest352-21runrttasks18:56:410
640991023,36cyclictest1417-21fschecks_time20:36:330
640991023,36cyclictest1417-21fschecks_time20:36:330
640991023,34cyclictest352-21runrttasks21:16:140
640991023,33cyclictest21496-21gpgv23:06:060
6409910220,28cyclictest3600-21switchtime19:01:410
2455221020,7sleep30-21swapper/323:11:053
640991013,40cyclictest352-21runrttasks22:06:560
640991003,37cyclictest947-21grep20:36:050
640991003,22cyclictest4431-21latency_hist23:31:110
640991003,22cyclictest4431-21latency_hist23:31:110
64099995,32cyclictest27369-21fschecks_time20:16:260
64099994,35cyclictest17375-21fschecks_count21:11:250
640999923,27cyclictest14056-21latency_hist19:36:060
64099992,25cyclictest7992-21cmp20:56:080
64099992,24cyclictest3184-21munin-plugin-st22:36:030
64099992,24cyclictest3184-21munin-plugin-st22:36:030
64099983,36cyclictest12051-21cstates21:56:200
64099983,35cyclictest29968-21fschecks_count21:32:240
64099983,33cyclictest10839-21latency_hist19:26:060
64099982,39cyclictest17155-21latency_hist19:46:060
64099982,35cyclictest5972-21basename20:51:030
657999768,20cyclictest15171-21unixbench_singl19:36:412
657999768,20cyclictest15171-21unixbench_singl19:36:412
64099964,34cyclictest6103-21latency_hist19:11:070
64099962,44cyclictest4639-21cstates23:31:230
64099953,35cyclictest17644-21fschecks_count19:46:230
200072950,6sleep30-21swapper/322:09:213
64099944,39cyclictest20405-21sh19:56:030
650999165,8cyclictest181rcu_preempt19:31:231
64099913,34cyclictest28714-21/usr/sbin/munin20:21:240
664999020,27cyclictest0-21swapper/323:26:203
664999020,27cyclictest0-21swapper/323:26:203
657999059,22cyclictest5313-21switchtime23:31:402
650998965,8cyclictest181rcu_preempt22:30:531
65799876,68cyclictest21760-21apt-get23:06:232
650998749,7cyclictest181rcu_preempt21:01:051
290902870,17sleep2371rcuc/221:31:062
657998615,60cyclictest28815-21apt-get21:32:012
650998665,15cyclictest2509-21sh20:41:041
650998665,15cyclictest2509-21sh20:41:041
650998541,12cyclictest181rcu_preempt20:01:291
650998541,11cyclictest23497-21ssh00:01:421
650998469,8cyclictest0-21swapper/122:41:261
650998469,8cyclictest0-21swapper/122:41:261
650998442,12cyclictest0-21swapper/122:25:031
650998438,11cyclictest0-21swapper/122:01:351
650998438,11cyclictest0-21swapper/122:01:351
650998437,9cyclictest181rcu_preempt23:46:241
657998360,16cyclictest25444-21latency_hist00:06:072
650998336,16cyclictest0-21swapper/119:01:441
65099832,42cyclictest31-21ksoftirqd/121:35:431
657998263,13cyclictest462-21/usr/sbin/munin22:31:402
657998263,13cyclictest462-21/usr/sbin/munin22:31:402
657998260,15cyclictest618-21cyclictest23:56:182
657998256,19cyclictest1257-21apt-key23:26:062
657998256,19cyclictest1257-21apt-key23:26:062
650998252,11cyclictest181rcu_preempt21:57:311
650998242,12cyclictest181rcu_preempt20:06:231
650998235,17cyclictest0-21swapper/122:06:431
657998167,9cyclictest618-21cyclictest19:06:262
657998167,9cyclictest618-21cyclictest19:06:262
657998159,15cyclictest618-21cyclictest22:53:002
657998156,16cyclictest2248-21cut00:21:242
657998156,16cyclictest2248-21cut00:21:242
650998151,12cyclictest181rcu_preempt00:21:391
650998151,12cyclictest181rcu_preempt00:21:391
650998150,14cyclictest181rcu_preempt19:07:181
650998150,14cyclictest181rcu_preempt19:07:181
650998143,16cyclictest181rcu_preempt00:11:261
650998143,16cyclictest181rcu_preempt00:11:261
650998057,8cyclictest181rcu_preempt22:53:521
650998046,13cyclictest181rcu_preempt21:11:041
650998042,8cyclictest181rcu_preempt21:13:141
650998041,9cyclictest0-21swapper/123:12:081
650998033,17cyclictest0-21swapper/122:17:291
640998055,19cyclictest2510-21latency00:21:290
640998055,19cyclictest2510-21latency00:21:290
64099803,41cyclictest13307-21cat23:46:060
657997959,14cyclictest11052-21awk23:41:312
657997950,21cyclictest2024-21sh23:26:552
657997950,21cyclictest2024-21sh23:26:552
657997950,11cyclictest8268-21awk23:36:392
657997950,11cyclictest8268-21awk23:36:392
65799794,32cyclictest16385-21ssh21:09:532
657997933,33cyclictest5880-21kworker/u19:2+nfsiod21:42:232
650997956,8cyclictest181rcu_preempt00:10:571
650997953,8cyclictest181rcu_preempt21:26:271
650997947,11cyclictest181rcu_preempt19:26:151
650997943,8cyclictest181rcu_preempt23:56:291
650997939,8cyclictest181rcu_preempt00:16:281
650997932,11cyclictest181rcu_preempt19:41:071
650997926,17cyclictest0-21swapper/121:56:041
650997926,17cyclictest0-21swapper/121:56:041
640997947,25cyclictest0-21swapper/000:01:280
657997860,11cyclictest4771-21entropy20:46:242
65799784,33cyclictest22214-21ssh22:12:102
657997822,11cyclictest0-21swapper/223:47:052
650997856,8cyclictest181rcu_preempt23:36:421
650997856,8cyclictest181rcu_preempt23:36:421
650997855,8cyclictest181rcu_preempt20:42:081
650997854,8cyclictest181rcu_preempt22:57:111
650997852,11cyclictest181rcu_preempt19:51:051
650997841,8cyclictest181rcu_preempt23:16:151
650997835,10cyclictest181rcu_preempt20:51:391
650997831,12cyclictest181rcu_preempt20:30:551
650997831,12cyclictest181rcu_preempt20:30:551
650997831,12cyclictest181rcu_preempt19:11:271
650997828,12cyclictest181rcu_preempt19:21:241
650997827,12cyclictest181rcu_preempt22:39:421
657997759,8cyclictest0-21swapper/221:51:052
657997759,13cyclictest7565-21latency_hist20:56:062
657997755,15cyclictest19356-21idleruntime19:51:262
650997751,10cyclictest181rcu_preempt22:33:531
650997751,10cyclictest181rcu_preempt22:33:531
650997746,13cyclictest181rcu_preempt19:16:381
650997734,12cyclictest181rcu_preempt21:36:281
650997729,16cyclictest0-21swapper/119:36:391
650997729,16cyclictest0-21swapper/119:36:391
64099773,40cyclictest352-21runrttasks22:43:010
64099773,40cyclictest352-21runrttasks22:43:010
657997656,12cyclictest21557-21kworker/u17:2-rpciod22:48:152
650997660,10cyclictest31338-21ntp_offset23:21:331
650997660,10cyclictest31338-21ntp_offset23:21:331
650997654,8cyclictest181rcu_preempt21:03:081
650997652,10cyclictest181rcu_preempt18:57:581
650997652,10cyclictest181rcu_preempt18:57:581
650997646,12cyclictest181rcu_preempt22:46:291
650997643,17cyclictest181rcu_preempt23:53:511
650997643,17cyclictest181rcu_preempt23:53:511
650997641,12cyclictest181rcu_preempt21:41:351
65099762,36cyclictest31-21ksoftirqd/122:14:431
64099765,40cyclictest27243-21fschecks_time22:21:240
650997543,10cyclictest181rcu_preempt20:21:251
650997538,11cyclictest181rcu_preempt21:25:011
640997542,12cyclictest181rcu_preempt23:59:480
657997427,34cyclictest27966-21grep22:21:382
657997415,20cyclictest17215-21cpuspeed21:11:202
650997451,9cyclictest181rcu_preempt23:01:301
650997451,8cyclictest181rcu_preempt19:56:211
650997442,12cyclictest181rcu_preempt21:16:261
650997433,10cyclictest0-21swapper/120:31:261
650997432,14cyclictest0-21swapper/120:51:051
657997356,11cyclictest3817-21ssh22:36:252
657997356,11cyclictest19570-21kworker/u18:1-rpciod21:21:062
657997347,15cyclictest0-21swapper/221:01:322
657997319,17cyclictest8128-21apt-get20:56:472
650997346,8cyclictest181rcu_preempt23:41:231
650997337,8cyclictest181rcu_preempt19:51:231
650997327,15cyclictest181rcu_preempt23:31:501
650997326,12cyclictest0-21swapper/121:46:221
64099733,39cyclictest352-21runrttasks23:00:300
657997255,10cyclictest30610-21latency_hist23:21:062
640997256,10cyclictest9664-21ssh22:46:230
640997248,14cyclictest352-21runrttasks21:26:060
640997233,28cyclictest618-21cyclictest19:07:560
64099723,30cyclictest4188-21processes22:36:350
657997151,14cyclictest618-21cyclictest20:27:262
657997151,14cyclictest618-21cyclictest20:27:262
657997148,13cyclictest16508-21rm23:51:162
657997148,13cyclictest16508-21rm23:51:162
657997145,13cyclictest0-21swapper/222:56:292
657997115,23cyclictest19721-21sh22:08:322
66499708,33cyclictest0-21swapper/320:42:033
65799707,23cyclictest2626-21touch20:41:052
65799707,23cyclictest2626-21touch20:41:052
657997047,12cyclictest0-21swapper/223:06:042
657997019,12cyclictest22516-21fschecks_time20:01:282
650997037,23cyclictest26845-21cron20:16:031
65799697,23cyclictest8411-21seq22:43:592
65799697,23cyclictest8411-21seq22:43:592
65799696,27cyclictest10192-21seq21:52:082
65799696,27cyclictest10192-21seq21:52:082
657996943,17cyclictest7579-21diskmemload22:03:182
657996943,17cyclictest7579-21diskmemload22:03:182
65799694,26cyclictest24933-21seq20:07:012
657996942,18cyclictest24188-21cut22:16:252
640996942,18cyclictest0-21swapper/020:16:050
65799683,47cyclictest9184-21latency_hist19:21:052
65799683,38cyclictest14087-21latency_hist19:36:062
65799682,44cyclictest28625-21latency_hist20:21:062
657996817,16cyclictest12051-21cstates21:56:202
64099684,38cyclictest15-21ksoftirqd/020:06:180
4702689,34sleep30-21swapper/318:56:033
65799674,33cyclictest352-21runrttasks20:15:322
65799674,25cyclictest12043-21seq19:27:272
657996719,10cyclictest31484-21apt-get00:16:142
657996717,12cyclictest0-21swapper/221:37:562
657996714,18cyclictest0-21swapper/221:20:282
66499663,40cyclictest917-21kworker/3:1+events21:01:383
65799665,27cyclictest26613-21sh00:06:422
657996645,12cyclictest0-21swapper/200:11:292
657996645,12cyclictest0-21swapper/200:11:292
657996643,12cyclictest0-21swapper/222:26:202
65799661,31cyclictest0-21swapper/223:12:492
64099663,43cyclictest15250-21grep22:01:280
64099663,43cyclictest15250-21grep22:01:280
65799655,39cyclictest352-21runrttasks19:58:422
65799654,39cyclictest9709-21kernelversion19:21:272
657996522,7cyclictest0-21swapper/220:21:272
640996544,14cyclictest23178-21grep20:01:420
64099653,41cyclictest31765-21latency_hist20:31:050
64099653,41cyclictest31765-21latency_hist20:31:050
55412650,9sleep3451rcuc/321:45:593
657996442,11cyclictest0-21swapper/218:56:402
657996442,11cyclictest0-21swapper/218:56:402
65799643,42cyclictest17966-21ntp_offset19:46:302
65799643,31cyclictest352-21runrttasks20:31:162
65799634,28cyclictest6769-21wc19:11:372
657996316,12cyclictest3307-21memory19:01:332
657996239,12cyclictest0-21swapper/220:42:062
66499604,35cyclictest917-21kworker/3:1+events20:36:343
66499604,35cyclictest917-21kworker/3:1+events20:36:343
664996020,30cyclictest0-21swapper/321:31:563
657996040,11cyclictest0-21swapper/219:46:062
66499593,39cyclictest6153-21taskset22:41:023
66499574,36cyclictest18906-21chrt21:13:333
66499564,33cyclictest9221-21chrt19:21:063
66499563,35cyclictest24540-21taskset00:04:193
66499555,33cyclictest31906-21taskset20:31:063
66499555,33cyclictest31906-21taskset20:31:063
66499553,37cyclictest15359-21chrt22:56:063
66499552,39cyclictest6940-21chrt21:46:403
66499552,36cyclictest1397-21chrt00:20:123
66499543,37cyclictest6357-21chrt19:11:263
66499543,37cyclictest23091-21chrt20:01:413
66499543,33cyclictest15868-21taskset19:41:193
66499542,38cyclictest10638-21chrt22:46:463
66499542,37cyclictest20716-21chrt23:57:023
66499533,38cyclictest9229-21chrt22:46:053
66499533,38cyclictest9229-21chrt22:46:053
66499533,37cyclictest25697-21chrt20:11:253
66499533,35cyclictest15107-21chrt22:01:233
66499533,35cyclictest15107-21chrt22:01:233
66499533,33cyclictest26461-21chrt21:26:253
66499532,34cyclictest21602-21chrt23:06:073
66499524,34cyclictest22805-21taskset21:21:023
66499523,36cyclictest12999-21chrt19:31:313
66499523,32cyclictest10994-21chrt21:54:223
66499523,32cyclictest10994-21chrt21:54:223
664995228,12cyclictest0-21swapper/300:11:373
664995228,12cyclictest0-21swapper/300:11:373
66499522,36cyclictest29902-21chrt22:26:113
66499522,34cyclictest6650-21chrt23:34:363
664995214,19cyclictest0-21swapper/320:56:183
66499513,34cyclictest32326-21taskset23:23:053
66499513,34cyclictest32326-21taskset23:23:053
66499513,33cyclictest917-21kworker/3:1+events23:53:303
66499513,33cyclictest917-21kworker/3:1+events23:53:303
66499513,31cyclictest917-21kworker/3:1+events22:11:133
66499513,31cyclictest15729-21chrt19:41:073
66499513,31cyclictest15729-21chrt19:41:073
66499512,35cyclictest14858-21taskset23:47:213
66499512,33cyclictest13373-21chrt21:57:383
66499512,33cyclictest12472-21chrt19:31:063
66499504,34cyclictest13213-21chrt23:46:033
66499503,33cyclictest698-21taskset22:31:263
66499503,33cyclictest698-21taskset22:31:263
66499503,33cyclictest16921-21chrt21:11:063
66499502,34cyclictest29333-21chrt23:18:023
66499494,33cyclictest26789-21chrt00:07:113
66499493,31cyclictest32591-21chrt21:36:273
66499492,34cyclictest17937-21chrt19:46:303
66499482,33cyclictest28518-21chrt20:21:053
66499482,30cyclictest7523-21taskset23:36:193
66499482,30cyclictest7523-21taskset23:36:193
66499474,30cyclictest2338-21taskset00:21:263
66499474,30cyclictest2338-21taskset00:21:263
66499473,33cyclictest27270-21chrt22:21:253
66499473,31cyclictest29487-21chrt20:21:383
66499472,32cyclictest25456-21chrt22:17:523
66499453,31cyclictest24706-21chrt23:11:103
66499443,30cyclictest20519-21chrt19:56:063
66499443,28cyclictest3005-21taskset19:01:253
66499442,32cyclictest16134-21taskset22:56:353
66499434,27cyclictest9935-21chrt19:21:343
66499433,29cyclictest21460-21chrt19:56:393
664994327,8cyclictest68802sleep320:51:363
66499422,24cyclictest917-21kworker/3:1+events20:46:143
66499412,25cyclictest1738-21taskset18:56:433
66499412,25cyclictest1738-21taskset18:56:433
66499403,27cyclictest32216-21taskset20:31:273
66499393,27cyclictest24851-21chrt20:06:393
66499393,27cyclictest23123-21taskset21:21:153
66499392,27cyclictest4651-21chrt19:06:233
66499392,27cyclictest4651-21chrt19:06:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional