You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-08 - 23:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Thu Jan 08, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
590621650,21sleep30-21swapper/307:05:053
7044991233,66cyclictest8081-21apt-get12:34:502
7044991133,69cyclictest29672-21apt-get10:29:502
7044991133,69cyclictest29672-21apt-get10:29:502
7036991113,81cyclictest7535-21apt-get12:34:481
7044991083,82cyclictest21111-21ssh09:20:092
70229910524,41cyclictest352-21runrttasks11:26:540
7036991023,74cyclictest16182-21sh11:55:161
3029621000,8sleep20-21swapper/208:20:072
2404821000,10sleep2371rcuc/210:20:052
7044999847,11cyclictest181rcu_preempt11:02:182
7044999847,11cyclictest181rcu_preempt11:02:182
7044999765,10cyclictest181rcu_preempt11:44:542
7022999631,28cyclictest2893-21ssh10:39:210
7044999573,13cyclictest0-21swapper/211:55:242
7022999534,36cyclictest0-21swapper/012:24:470
7022999528,27cyclictest22968-21rm09:24:260
7022999430,18cyclictest25430-21ssh11:16:260
7022999430,18cyclictest25430-21ssh11:16:260
7022999428,26cyclictest28313-21sh10:27:200
7022999331,27cyclictest9620-21ssh12:36:220
7022999330,31cyclictest26757-21/usr/sbin/munin09:29:560
7022999330,29cyclictest397-21ssh12:21:380
7022999328,28cyclictest8532-21ssh11:42:170
7022999327,29cyclictest20082-21/usr/sbin/munin12:00:090
7022999326,26cyclictest25861-21/usr/sbin/munin12:10:060
7022999325,26cyclictest0-21swapper/012:05:520
702299929,45cyclictest0-21swapper/009:41:280
7022999229,28cyclictest13032-21ssh10:02:030
7022999229,28cyclictest13032-21ssh10:02:030
7022999229,24cyclictest3083-21rm11:34:040
7022999228,28cyclictest30849-21ssh12:19:310
7022999228,26cyclictest12716-21ssh10:54:540
7022999225,42cyclictest352-21runrttasks10:20:260
7022999225,34cyclictest14447-21kworker/u17:3+rpciod10:47:180
601229221,36sleep20-21swapper/207:05:092
7022999133,45cyclictest29286-21awk08:19:330
7022999127,23cyclictest352-21runrttasks07:14:480
7022999125,38cyclictest0-21swapper/009:47:230
7044999048,10cyclictest181rcu_preempt11:39:582
7044999038,11cyclictest181rcu_preempt10:36:022
7022999031,22cyclictest10020-21sh11:44:550
7022999028,27cyclictest29826-21rm10:30:070
7022999028,27cyclictest29826-21rm10:30:070
7022999024,29cyclictest0-21swapper/011:02:100
7022999024,29cyclictest0-21swapper/011:02:100
7044998935,13cyclictest181rcu_preempt11:09:492
7044998932,8cyclictest0-21swapper/211:31:372
703699893,61cyclictest29946-21ssh10:30:091
703699893,61cyclictest29946-21ssh10:30:091
7022998931,22cyclictest19047-21seq10:11:560
7022998929,31cyclictest3502-21sh10:39:510
7022998929,27cyclictest24586-21seq09:25:320
7022998929,25cyclictest13218-21diskmemload10:04:510
7022998928,20cyclictest15627-21seq07:34:000
7022998927,28cyclictest11953-21ssh10:54:080
7022998927,27cyclictest7394-21ssh12:34:130
7022998927,26cyclictest17524-21ssh11:56:210
7022998926,22cyclictest32406-21seq08:28:400
7022998925,28cyclictest6383-21/usr/sbin/munin08:45:240
7022998925,28cyclictest6383-21/usr/sbin/munin08:45:240
7022998924,26cyclictest11238-21/usr/sbin/munin09:00:110
704999882,47cyclictest8302-21chrt12:35:223
7022998835,20cyclictest19557-21ssh09:19:140
7022998829,28cyclictest23051-21ssh11:13:140
7022998829,26cyclictest10031-21ssh09:57:130
7022998828,26cyclictest20618-21ssh10:15:020
7022998825,23cyclictest13019-21/usr/sbin/munin11:49:540
562228824,34sleep00-21swapper/007:04:540
7044998758,8cyclictest181rcu_preempt09:35:442
7044998758,8cyclictest181rcu_preempt09:35:442
7044998742,13cyclictest181rcu_preempt07:49:542
7044998735,13cyclictest181rcu_preempt08:24:552
7022998731,19cyclictest27662-21unin-run08:14:340
7022998730,28cyclictest30987-21ssh09:36:260
7022998730,28cyclictest30987-21ssh09:36:260
7022998729,23cyclictest6980-21/usr/sbin/munin08:49:550
7022998729,20cyclictest3789-21taskset08:39:490
7022998729,20cyclictest3789-21taskset08:39:490
7022998728,20cyclictest7348-21ssh09:50:330
7022998728,20cyclictest7348-21ssh09:50:330
7022998728,20cyclictest20275-21taskset07:46:230
7022998727,24cyclictest19605-21ssh11:07:060
7022998726,25cyclictest15387-21ssh09:12:120
7022998718,30cyclictest29823-21latency_hist11:24:350
7044998638,17cyclictest181rcu_preempt12:25:052
7044998638,12cyclictest181rcu_preempt07:31:202
704499863,64cyclictest16796-21apt-get09:14:512
7036998667,13cyclictest23168-21latency_hist09:24:361
7022998634,23cyclictest3596-21ssh11:34:450
7022998629,19cyclictest10009-21systemd-run08:59:340
7022998627,24cyclictest12645-21df07:24:520
7022998627,24cyclictest12645-21df07:24:520
7022998626,21cyclictest0-21swapper/008:29:480
152602863,33sleep114721-21/usr/sbin/munin10:59:571
152602863,33sleep114721-21/usr/sbin/munin10:59:571
7044998559,10cyclictest181rcu_preempt10:11:572
7022998529,20cyclictest20803-21runrttasks07:49:470
7022998527,23cyclictest12939-21seq09:06:190
7022998527,23cyclictest12939-21seq09:06:190
7022998520,30cyclictest24128-21awk07:59:520
7044998448,13cyclictest181rcu_preempt07:49:262
7036998468,11cyclictest7013-21cyclictest11:14:551
7036998468,11cyclictest7013-21cyclictest11:14:551
7022998426,22cyclictest17238-21taskset07:39:050
7022998426,22cyclictest17238-21taskset07:39:050
7022998423,23cyclictest11222-21/usr/sbin/munin07:19:570
7022998418,29cyclictest8371-21chrt07:12:260
7022998418,29cyclictest8371-21chrt07:12:260
7044998340,17cyclictest181rcu_preempt10:08:312
7044998337,13cyclictest181rcu_preempt11:36:462
7044998310,33cyclictest30240-21grep11:24:542
7036998363,14cyclictest3356-21latency_hist11:34:351
7022998325,20cyclictest18650-21taskset07:41:100
7022998325,20cyclictest18650-21taskset07:41:100
7044998246,12cyclictest181rcu_preempt09:03:562
7044998243,8cyclictest181rcu_preempt07:19:502
7044998237,15cyclictest181rcu_preempt12:09:482
7044998231,11cyclictest181rcu_preempt12:09:062
7044998153,19cyclictest298-21in:imuxsock11:54:322
7044998151,9cyclictest181rcu_preempt11:09:352
7044998145,11cyclictest181rcu_preempt10:54:362
7044998144,13cyclictest181rcu_preempt10:26:302
7044998143,20cyclictest181rcu_preempt09:04:492
7044998143,20cyclictest181rcu_preempt09:04:492
7044998140,10cyclictest181rcu_preempt10:00:052
7044998140,10cyclictest181rcu_preempt10:00:052
7022998128,24cyclictest26042-21modprobe08:09:340
7044998052,9cyclictest181rcu_preempt09:40:262
7044998049,8cyclictest181rcu_preempt12:29:492
7044998048,9cyclictest181rcu_preempt08:38:032
7036998061,14cyclictest26139-21latency_hist08:09:361
7036998059,14cyclictest10819-21latency_hist07:19:371
7022998026,20cyclictest2955-21taskset08:35:500
7044997940,11cyclictest181rcu_preempt10:44:512
7036997959,13cyclictest300-21rs:main1
7036997950,21cyclictest5225-21sh12:29:571
703699792,37cyclictest31-21ksoftirqd/111:22:531
7022997916,28cyclictest23011-21ps07:55:050
7044997841,13cyclictest181rcu_preempt12:00:552
704499783,44cyclictest29110-21timerwakeupswit12:15:082
7036997859,13cyclictest5757-21kernelversion08:45:121
7036997859,13cyclictest5757-21kernelversion08:45:121
7036997829,11cyclictest0-21swapper/108:35:091
7022997818,25cyclictest30015-21/usr/sbin/munin08:20:000
7044997744,11cyclictest181rcu_preempt10:56:412
7044997743,11cyclictest181rcu_preempt08:00:112
7044997738,9cyclictest181rcu_preempt11:19:362
7044997738,9cyclictest181rcu_preempt11:19:362
7044997733,11cyclictest181rcu_preempt12:24:352
7044997726,23cyclictest0-21swapper/208:40:522
7044997726,23cyclictest0-21swapper/208:40:522
7036997754,17cyclictest32156-21idleruntime-cro09:39:331
7036997754,17cyclictest32156-21idleruntime-cro09:39:331
7036997753,17cyclictest22216-21latency_hist07:54:351
7036997752,15cyclictest200-21systemd-journal12:03:261
703699772,57cyclictest9742-21kworker/u17:2+ext4-rsv-conversion09:19:391
7044997653,9cyclictest181rcu_preempt07:59:352
7044997650,9cyclictest181rcu_preempt10:44:212
7044997645,13cyclictest181rcu_preempt07:15:082
7044997642,13cyclictest181rcu_preempt07:40:052
7044997642,13cyclictest181rcu_preempt07:40:052
7044997641,13cyclictest181rcu_preempt08:14:352
7044997640,9cyclictest181rcu_preempt09:47:362
7044997628,26cyclictest0-21swapper/208:49:482
7036997661,8cyclictest0-21swapper/110:54:331
7044997545,13cyclictest181rcu_preempt08:15:062
7044997540,13cyclictest181rcu_preempt10:15:362
7044997539,13cyclictest181rcu_preempt08:09:372
7044997530,12cyclictest0-21swapper/209:50:082
7044997530,12cyclictest0-21swapper/209:50:082
7036997561,10cyclictest7013-21cyclictest09:52:031
7036997561,10cyclictest7013-21cyclictest09:52:031
7036997561,10cyclictest7013-21cyclictest09:10:091
7036997557,12cyclictest8550-21latency_hist07:14:361
7036997557,12cyclictest8550-21latency_hist07:14:361
7044997450,9cyclictest181rcu_preempt09:57:572
7044997450,9cyclictest181rcu_preempt08:55:092
7044997448,10cyclictest181rcu_preempt11:21:042
704499743,51cyclictest13218-21diskmemload09:14:362
7036997459,11cyclictest7013-21cyclictest12:14:551
7036997436,10cyclictest58-21kcompactd011:36:131
7036997411,31cyclictest31-21ksoftirqd/110:18:151
704999732,57cyclictest16019-21chrt11:54:493
7044997345,11cyclictest181rcu_preempt07:35:012
7044997345,11cyclictest181rcu_preempt07:35:012
7044997341,8cyclictest181rcu_preempt09:30:072
7044997339,12cyclictest181rcu_preempt07:14:362
7044997339,12cyclictest181rcu_preempt07:14:362
704499733,42cyclictest503-21fschecks_count08:29:562
7044997330,7cyclictest181rcu_preempt07:29:372
7044997330,7cyclictest181rcu_preempt07:29:372
7036997354,14cyclictest13218-21diskmemload12:09:341
7036997350,14cyclictest200-21systemd-journal10:47:251
703699732,54cyclictest29938-21kworker/u17:1+ext4-rsv-conversion10:54:491
7036997311,32cyclictest31-21ksoftirqd/111:42:471
7036997310,26cyclictest0-21swapper/112:25:031
7044997245,12cyclictest181rcu_preempt08:45:192
7044997245,12cyclictest181rcu_preempt08:45:192
7036997259,9cyclictest2-21kthreadd08:04:341
7036997254,13cyclictest12229-21idleruntime-cro07:24:331
7036997254,12cyclictest200-21systemd-journal09:44:321
7036997248,18cyclictest12920-21entropy11:49:521
7036997244,18cyclictest1-21systemd09:34:351
7036997235,10cyclictest10608-21kworker/u19:0-rpciod11:10:261
7036997232,29cyclictest32480-21modprobe08:29:331
703699722,59cyclictest352-21runrttasks07:29:491
7036997212,30cyclictest31-21ksoftirqd/110:27:451
7044997139,13cyclictest181rcu_preempt09:29:112
7036997149,11cyclictest0-21swapper/108:24:361
7036997136,9cyclictest7013-21cyclictest12:10:571
7036997136,10cyclictest30624-21kworker/u19:2-rpciod11:28:191
7036997136,10cyclictest25839-21kworker/u19:1-nfsiod09:59:161
7036997134,10cyclictest7013-21cyclictest07:29:181
7036997134,10cyclictest7013-21cyclictest07:29:181
7036997047,15cyclictest200-21systemd-journal11:05:251
7036997036,10cyclictest4611-21ssh10:40:411
7036996952,13cyclictest5746-21kworker/u17:1-rpciod10:19:551
7036996936,19cyclictest7013-21cyclictest10:01:041
7036996936,19cyclictest7013-21cyclictest10:01:041
703699689,32cyclictest31-21ksoftirqd/110:14:091
7036996854,8cyclictest0-21swapper/109:44:531
7036996853,8cyclictest0-21swapper/108:14:351
7036996851,11cyclictest297-21systemd-logind07:44:341
7036996851,11cyclictest297-21systemd-logind07:44:341
7036996844,7cyclictest181rcu_preempt08:43:451
7036996844,7cyclictest181rcu_preempt08:43:451
7036996832,9cyclictest7013-21cyclictest09:03:181
703699674,24cyclictest31183-21/usr/sbin/munin12:20:051
7036996733,10cyclictest58-21kcompactd009:05:441
7036996733,10cyclictest58-21kcompactd009:05:441
7036996733,10cyclictest10037-21kworker/u19:4-events_unbound07:45:241
7036996645,9cyclictest181rcu_preempt08:55:251
7036996636,15cyclictest15739-21ssh10:06:091
7036996635,9cyclictest58-21kcompactd011:46:461
7036996623,22cyclictest31-21ksoftirqd/110:34:511
703699662,31cyclictest17301-21ls07:39:341
703699662,31cyclictest17301-21ls07:39:341
703699653,35cyclictest28065-21fschecks_count08:14:551
7049996442,14cyclictest451rcuc/307:11:533
7049996442,14cyclictest451rcuc/307:11:533
66272647,37sleep10-21swapper/107:06:181
7036996330,9cyclictest21645-21kworker/u19:2-flush-0:3209:26:271
7049996250,8cyclictest451rcuc/307:54:303
704999623,24cyclictest0-21swapper/310:30:093
704999623,24cyclictest0-21swapper/310:30:093
704999623,20cyclictest0-21swapper/309:19:383
703699624,32cyclictest1648-21latency_hist08:34:371
7036996136,7cyclictest181rcu_preempt07:56:181
7049995743,10cyclictest451rcuc/307:37:123
7049995743,10cyclictest451rcuc/307:37:123
704999573,41cyclictest32424-21taskset09:39:373
704999572,31cyclictest20310-21chrt09:19:443
7049995543,8cyclictest451rcuc/311:30:223
7049995542,9cyclictest451rcuc/307:19:223
704999553,42cyclictest24483-21taskset12:08:433
7049995440,10cyclictest451rcuc/308:39:213
7049995439,10cyclictest451rcuc/308:13:373
7049995436,11cyclictest38752sleep310:40:003
7049995340,9cyclictest451rcuc/308:46:473
7049995340,9cyclictest451rcuc/308:46:473
7049995338,10cyclictest451rcuc/308:21:353
704999533,40cyclictest7072-21chrt12:33:193
704999533,40cyclictest32150-21chrt09:39:333
704999533,40cyclictest32150-21chrt09:39:333
704999532,34cyclictest0-21swapper/310:54:493
115422530,6sleep30-21swapper/309:59:513
115422530,6sleep30-21swapper/309:59:513
704999524,35cyclictest17181-21taskset10:09:363
7049995240,8cyclictest451rcuc/307:59:523
7049995239,9cyclictest451rcuc/308:04:423
704999523,39cyclictest1675-21taskset08:34:343
7049995138,9cyclictest451rcuc/307:47:543
7049995138,9cyclictest451rcuc/307:27:043
7049995138,9cyclictest451rcuc/307:27:043
7049995137,10cyclictest451rcuc/307:21:333
704999513,39cyclictest27963-21chrt09:31:273
704999512,40cyclictest14127-21taskset09:10:023
704999504,33cyclictest451-21taskset11:29:363
7049995037,9cyclictest451rcuc/309:01:333
7049995037,9cyclictest451rcuc/307:29:553
7049995036,9cyclictest451rcuc/312:04:063
704999503,36cyclictest20278-21taskset10:14:393
704999493,37cyclictest15342-21taskset11:54:203
704999493,36cyclictest9596-21taskset08:55:123
704999493,36cyclictest25309-21chrt12:09:523
704999493,34cyclictest17616-21chrt11:04:363
704999493,34cyclictest17616-21chrt11:04:363
704999493,33cyclictest32650-21chrt08:29:363
704999484,35cyclictest8167-21taskset10:47:053
7049994835,9cyclictest451rcuc/308:43:023
7049994835,9cyclictest451rcuc/308:43:023
704999483,37cyclictest4345-21taskset12:29:093
704999483,35cyclictest3881-21chrt11:34:553
704999483,35cyclictest28284-21chrt10:27:153
704999483,35cyclictest1464-21chrt10:35:263
704999483,32cyclictest1642-21chrt12:24:353
704999482,37cyclictest25798-21chrt09:28:513
704999482,37cyclictest25499-21chrt11:16:353
704999482,37cyclictest25499-21chrt11:16:353
704999482,34cyclictest30709-21chrt12:19:083
704999473,35cyclictest6270-21taskset09:49:513
704999473,35cyclictest6270-21taskset09:49:513
704999473,35cyclictest27812-21chrt11:20:083
704999473,33cyclictest9499-21chrt10:49:483
704999473,33cyclictest9232-21taskset09:55:013
704999463,32cyclictest24112-21chrt10:20:063
704999462,35cyclictest5348-21chrt09:48:253
704999462,35cyclictest19363-21chrt10:12:483
704999462,33cyclictest12324-21chrt11:49:223
704999454,28cyclictest23380-21chrt07:55:103
704999453,33cyclictest19339-21chrt11:06:223
7049994532,9cyclictest451rcuc/309:08:013
7049994532,9cyclictest451rcuc/309:08:013
7049994531,9cyclictest451rcuc/307:43:443
7049994531,9cyclictest451rcuc/307:43:443
7049994530,10cyclictest451rcuc/311:10:123
704999452,33cyclictest7644-21chrt08:50:063
704999452,33cyclictest6880-21chrt11:39:543
704999423,26cyclictest27899-21taskset08:14:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional