You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 07:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Mon Dec 22, 2025 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245029911910,82cyclictest24451-21apt-get21:50:152
1366921180,15sleep239-21ksoftirqd/200:10:222
244959910664,33cyclictest0-21swapper/021:40:240
24502991043,53cyclictest18161-21ssh23:25:122
24502991033,29cyclictest346-21runrttasks20:55:012
24502991014,36cyclictest4579-21fschecks_count22:10:232
24499999946,46cyclictest16605-21idleruntime-cro22:30:001
24499999946,46cyclictest16605-21idleruntime-cro22:30:001
24499999935,12cyclictest0-21swapper/123:50:001
2450299962,27cyclictest346-21runrttasks19:49:272
2449999954,38cyclictest6194-21fschecks_count23:05:211
2449999952,70cyclictest13603-21apt-get00:10:351
24502999457,30cyclictest382-21rs:main2
24502999457,30cyclictest382-21rs:main2
24502999457,30cyclictest382-21rs:main2
24499999443,7cyclictest0-21swapper/121:20:221
24499999438,12cyclictest0-21swapper/121:15:181
24499999438,12cyclictest0-21swapper/121:15:181
2449599943,76cyclictest13852-21apt-get22:25:150
2449599943,76cyclictest13852-21apt-get22:25:150
2450299936,76cyclictest10325-21apt-get00:05:132
2450299936,76cyclictest10325-21apt-get00:05:132
24499999237,12cyclictest0-21swapper/119:45:231
2449599922,74cyclictest13660-21ssh00:10:200
24499999142,10cyclictest21806-21proc_pri21:45:331
2449999913,44cyclictest7138-21systemd-run22:15:011
24502999046,36cyclictest0-21swapper/219:56:492
24502999046,36cyclictest0-21swapper/219:56:492
24499999039,13cyclictest7279-21fschecks_count00:00:231
24499999039,13cyclictest7279-21fschecks_count00:00:231
2450299892,35cyclictest32571-21fschecks_count22:55:242
2450299892,25cyclictest346-21runrttasks00:23:452
24499998962,18cyclictest19235-21timerandwakeup21:41:251
24495998961,8cyclictest181rcu_preempt23:44:030
24502998870,12cyclictest7753-21fschecks_time22:15:262
24502998866,11cyclictest4715-21cut23:55:332
24502998863,16cyclictest24488-21cyclictest21:35:262
24502998852,24cyclictest17709-21head00:15:392
2450299883,32cyclictest24005-21mailstats20:45:352
2450299883,32cyclictest23244-21cat20:45:032
2450299883,32cyclictest23244-21cat20:45:032
2449999883,36cyclictest31985-21grep21:10:231
2449999883,36cyclictest31985-21grep21:10:231
2449599883,67cyclictest24702-21apt-get20:50:140
24502998765,16cyclictest11709-21sh23:15:002
2450299876,29cyclictest15427-21rm22:26:532
2450299876,29cyclictest15427-21rm22:26:532
24502998758,22cyclictest23923-21apt-key21:50:012
2450299873,36cyclictest28985-21ssh22:50:012
24495998751,10cyclictest181rcu_preempt22:21:110
24495998739,18cyclictest181rcu_preempt22:17:510
2450299864,38cyclictest20184-21/usr/sbin/munin22:35:192
2450299863,36cyclictest28521-21awk00:34:592
2450299863,34cyclictest12214-21ssh21:30:292
24499998631,11cyclictest0-21swapper/122:30:361
24495998656,20cyclictest28261-21latency_hist19:20:040
24495998640,29cyclictest181rcu_preempt21:31:180
2450299853,35cyclictest2996-21ssh21:15:322
2450299853,35cyclictest2996-21ssh21:15:322
2450299853,34cyclictest15305-21sh23:20:202
2450299853,33cyclictest719-21/usr/sbin/munin19:35:122
2450299853,31cyclictest31741-21ssh21:10:132
2450299853,31cyclictest31741-21ssh21:10:132
2450299852,28cyclictest4671-21rm23:02:222
24495998531,27cyclictest31963-21fschecks_count21:10:230
24495998531,27cyclictest31963-21fschecks_count21:10:230
242922856,65sleep20-21swapper/219:09:122
2450299844,37cyclictest30016-21latency_hist23:45:032
2450299844,33cyclictest17798-21grep20:25:362
2450299843,36cyclictest31412-21modprobe21:10:002
2450299843,35cyclictest31345-21unin-run19:30:012
2450299843,35cyclictest31345-21unin-run19:30:012
2450299843,32cyclictest1913-21sh22:05:332
2450299842,9cyclictest0-21swapper/219:50:132
24495998466,12cyclictest320-21dbus-daemon00:05:010
24495998466,12cyclictest320-21dbus-daemon00:05:010
2435328412,44sleep124379-21munin-plugin-st19:10:011
2450299834,33cyclictest5943-21/usr/sbin/munin23:05:112
2450299833,34cyclictest15777-21ntp_kernel_err20:20:282
2450299833,33cyclictest25645-21latency_hist00:30:022
2450299833,33cyclictest25645-21latency_hist00:30:022
2450299833,28cyclictest27229-21latency20:55:282
2450299833,22cyclictest7051-21/usr/sbin/munin00:00:122
2450299833,22cyclictest7051-21/usr/sbin/munin00:00:122
2450299833,10cyclictest0-21swapper/223:35:172
2450299832,31cyclictest25869-21latency_hist22:45:032
2450299832,31cyclictest25869-21latency_hist22:45:032
24495998369,8cyclictest8527-21kworker/u17:020:00:000
24495998369,8cyclictest8527-21kworker/u17:020:00:000
24495998367,11cyclictest24488-21cyclictest23:49:020
24495998353,13cyclictest181rcu_preempt19:20:160
24495998350,11cyclictest181rcu_preempt23:10:100
24495998330,17cyclictest181rcu_preempt21:20:390
2450299823,36cyclictest14990-21modprobe20:20:002
2450299823,34cyclictest31146-21/usr/sbin/munin22:00:262
2450299823,34cyclictest29453-21ntp_states00:35:312
2450299823,34cyclictest29453-21ntp_states00:35:312
2450299823,30cyclictest31320-21timerwakeupswit23:45:392
24499998258,17cyclictest14927-21latency_hist23:20:031
24495998265,11cyclictest30979-21sh22:00:230
24495998263,13cyclictest358-21ntpd00:06:090
24495998263,13cyclictest358-21ntpd00:06:090
24502998155,18cyclictest10597-21sh21:28:072
2450299813,36cyclictest13453-21apt-config22:25:022
2450299813,34cyclictest20985-21latency_hist21:45:032
2450299813,33cyclictest12005-21/usr/sbin/munin23:15:242
2450299813,32cyclictest27998-21latency21:55:252
2450299813,27cyclictest30551-21ssh22:51:092
2450299813,27cyclictest10703-21fschecks_time20:05:222
2450299812,21cyclictest12722-21grep20:10:382
2449999814,36cyclictest27756-21head20:55:391
2449999812,63cyclictest24508-21sh21:50:201
24495998164,11cyclictest320-21dbus-daemon20:50:010
24495998161,14cyclictest31409-21diskmemload23:35:300
24495998122,34cyclictest15-21ksoftirqd/021:15:440
24495998122,34cyclictest15-21ksoftirqd/021:15:440
2450299809,30cyclictest17712-21ssh22:30:342
2450299803,35cyclictest22006-21expr23:30:362
2450299803,35cyclictest22006-21expr23:30:362
2450299803,33cyclictest24854-21/usr/sbin/munin19:10:212
2450299803,33cyclictest1399-21cat23:50:272
2450299803,33cyclictest1399-21cat23:50:272
24502998029,45cyclictest0-21swapper/219:25:012
2450299802,36cyclictest27379-21mailstats19:15:362
2450299802,32cyclictest5876-21taskset21:20:232
2450299802,31cyclictest2804-21sed19:40:252
24499998051,20cyclictest392-21cron22:10:001
2449599802,33cyclictest25859-21modprobe22:45:000
2449599802,33cyclictest25859-21modprobe22:45:000
2450299793,8cyclictest0-21swapper/221:00:172
2450299793,34cyclictest19487-21/usr/sbin/munin20:30:372
2450299793,32cyclictest9006-21fschecks_count20:00:192
2450299792,27cyclictest20289-21fschecks_count20:35:252
24499997963,10cyclictest506-21open_inodes22:55:341
24499997957,15cyclictest29824-21latency_hist19:25:031
24495997959,14cyclictest14910-21latency_hist23:20:030
24495997958,11cyclictest32562-21ssh22:55:240
24495997942,13cyclictest181rcu_preempt20:40:230
24495997942,13cyclictest181rcu_preempt20:40:230
24499997833,12cyclictest346-21runrttasks20:05:031
24499997818,47cyclictest9776-21apt-get00:05:131
24499997818,47cyclictest9776-21apt-get00:05:131
2449599787,27cyclictest6945-21sed00:00:050
24495997858,11cyclictest17651-21latency_hist21:40:020
24495997856,13cyclictest24488-21cyclictest21:59:230
2449599783,48cyclictest15-21ksoftirqd/000:17:330
24499997760,9cyclictest1166-21kworker/u18:0+flush-179:800:30:031
24495997755,17cyclictest24488-21cyclictest21:45:240
24495997748,11cyclictest181rcu_preempt22:13:230
24495997737,32cyclictest0-21swapper/022:55:030
24499997658,12cyclictest21512-21kworker/u17:1-xprtiod00:35:371
24499997658,12cyclictest21512-21kworker/u17:1-xprtiod00:35:371
2449999762,40cyclictest7763-21ps19:55:341
2449999762,40cyclictest7763-21ps19:55:341
2449599764,28cyclictest346-21runrttasks22:07:540
24495997625,12cyclictest181rcu_preempt22:49:090
240502765,53sleep00-21swapper/019:06:000
24499997557,12cyclictest346-21runrttasks20:45:341
24499997555,14cyclictest329-21systemd-logind20:40:011
24499997549,8cyclictest181rcu_preempt22:50:351
24499997525,21cyclictest3062-21cstates23:00:181
24495997558,12cyclictest31409-21diskmemload21:55:020
24495997557,11cyclictest31553-21sendmail00:40:000
24495997557,11cyclictest31553-21sendmail00:40:000
2449599752,39cyclictest24134-21chrt00:26:240
2449599752,39cyclictest24134-21chrt00:26:240
24499997460,9cyclictest28793-21kworker/u17:0-rpciod22:25:001
24499997455,13cyclictest11916-21latency_hist20:10:031
24495997456,12cyclictest24488-21cyclictest20:14:400
24495997453,14cyclictest60-21kcompactd020:05:340
24495997450,11cyclictest181rcu_preempt20:04:220
24495997445,9cyclictest181rcu_preempt19:40:280
24495997435,11cyclictest181rcu_preempt19:30:120
24495997435,11cyclictest181rcu_preempt19:30:120
24495997435,11cyclictest181rcu_preempt19:30:120
24495997424,8cyclictest181rcu_preempt20:39:040
24499997352,12cyclictest0-21swapper/119:15:251
24495997357,10cyclictest31409-21diskmemload23:55:020
24495997357,10cyclictest31409-21diskmemload23:55:020
2449599734,47cyclictest20014-21fschecks_time00:20:240
24495997336,17cyclictest181rcu_preempt21:28:030
2449599733,40cyclictest15-21ksoftirqd/019:35:230
24499997251,7cyclictest181rcu_preempt21:31:261
24499997251,14cyclictest346-21runrttasks19:55:041
24495997258,10cyclictest24488-21cyclictest23:22:460
24495997250,15cyclictest3525-21ntp_offset23:00:290
2449599723,46cyclictest26276-21latency00:30:260
24499997152,13cyclictest13894-21cut20:15:241
24499997132,22cyclictest18371-21fschecks_count23:25:211
24495997140,22cyclictest16143-21grep20:20:360
2449599712,38cyclictest19198-21grep23:25:360
24499997049,16cyclictest17904-21latency_hist23:25:021
24499997044,17cyclictest1-21systemd19:40:581
24495997052,13cyclictest316-21kworker/u17:3+xprtiod22:30:370
24495997045,12cyclictest181rcu_preempt20:27:440
24495997045,10cyclictest181rcu_preempt23:07:220
24495997044,10cyclictest181rcu_preempt20:30:230
2449599703,54cyclictest24891-21fschecks_count19:10:220
24499996954,8cyclictest0-21swapper/123:35:261
24499996933,9cyclictest6042-21kworker/u19:1-flush-179:823:43:371
24499996928,8cyclictest181rcu_preempt19:10:361
2449999692,39cyclictest11536-21ssh23:14:331
24495996947,16cyclictest60-21kcompactd022:38:560
2449599694,30cyclictest22015-21switchtime23:30:360
2449599694,30cyclictest22015-21switchtime23:30:360
24495996929,30cyclictest0-21swapper/019:45:180
24499996850,13cyclictest24488-21cyclictest21:55:341
24499996849,14cyclictest31589-21cpu19:30:171
24499996849,14cyclictest31589-21cpu19:30:171
24499996849,14cyclictest31589-21cpu19:30:171
24499996846,16cyclictest1-21systemd20:42:581
24499996846,16cyclictest1-21systemd20:42:581
24499996841,19cyclictest201-21systemd-journal23:59:571
24499996820,22cyclictest3896-21date23:55:031
24499996817,13cyclictest181rcu_preempt22:40:221
24499996817,13cyclictest181rcu_preempt22:40:221
24495996840,12cyclictest181rcu_preempt21:00:230
24495996831,13cyclictest181rcu_preempt21:05:250
24499996740,20cyclictest23074-21fschecks_time00:25:251
2449999673,49cyclictest12458-21grep20:10:301
24499996720,13cyclictest15234-21sed21:35:301
2449599673,38cyclictest15-21ksoftirqd/020:16:340
24511996616,33cyclictest0-21swapper/300:10:323
24499996648,13cyclictest26471-21fschecks_time22:45:221
24499996641,11cyclictest181rcu_preempt23:33:251
24499996641,11cyclictest181rcu_preempt23:33:251
24499996639,8cyclictest181rcu_preempt00:17:331
2449999662,42cyclictest26818-21unixbench_multi00:30:381
24495996653,9cyclictest60-21kcompactd019:54:440
24495996639,11cyclictest181rcu_preempt19:26:300
24495996639,11cyclictest181rcu_preempt19:26:300
24499996546,13cyclictest9293-21sendmail_mailst21:25:351
2449999654,44cyclictest22937-21latency_hist22:40:031
2449599652,43cyclictest15-21ksoftirqd/020:55:120
24499996444,14cyclictest60-21kcompactd020:24:371
24499996443,15cyclictest26555-21cut20:55:021
24499996442,15cyclictest301ktimers/121:00:171
24499996343,12cyclictest346-21runrttasks22:19:381
2449999633,41cyclictest31049-21kernelversion22:00:251
2449999622,45cyclictest20031-21fschecks_time00:20:241
241872625,13sleep30-21swapper/319:07:513
24499996140,14cyclictest0-21swapper/119:25:241
24499996140,14cyclictest0-21swapper/119:25:241
2449999613,39cyclictest346-21runrttasks20:32:251
24499996127,8cyclictest181rcu_preempt23:50:171
2449999612,38cyclictest31493-21latency_hist21:10:031
2451199597,34cyclictest0-21swapper/322:25:143
2451199597,34cyclictest0-21swapper/322:25:143
24499995942,12cyclictest0-21swapper/119:40:041
2449999593,38cyclictest18399-21cat20:30:041
2451199563,39cyclictest18135-21taskset21:40:243
2451199563,38cyclictest24393-21chrt21:50:053
2451199553,39cyclictest2192-21chrt19:40:033
2451199553,38cyclictest19610-21chrt23:26:363
24511995511,26cyclictest0-21swapper/320:50:143
2451199546,35cyclictest32168-21taskset00:40:333
2451199546,35cyclictest32168-21taskset00:40:333
2451199543,41cyclictest31426-21taskset19:30:023
2451199543,41cyclictest31426-21taskset19:30:023
2451199543,37cyclictest366-21taskset22:55:283
2451199543,37cyclictest3241-21chrt23:53:543
2451199543,37cyclictest3241-21chrt23:53:543
2451199543,37cyclictest23339-21chrt22:40:213
2451199543,37cyclictest23339-21chrt22:40:213
2451199535,36cyclictest29227-21taskset21:00:373
2451199534,35cyclictest19626-21chrt00:20:033
2451199533,39cyclictest30760-21chrt22:51:443
2451199533,38cyclictest0-21swapper/300:05:243
2451199533,38cyclictest0-21swapper/300:05:243
2451199533,36cyclictest6366-21chrt23:05:253
2451199532,39cyclictest8770-21chrt22:16:153
2451199532,38cyclictest9062-21taskset00:03:063
2451199532,38cyclictest9062-21taskset00:03:063
2451199524,35cyclictest5820-21taskset21:20:223
2451199524,32cyclictest5413-21chrt19:50:033
2451199523,38cyclictest30421-21taskset21:05:273
2451199523,38cyclictest11390-21taskset22:20:363
2451199523,37cyclictest7345-21taskset19:55:213
2451199523,37cyclictest7345-21taskset19:55:213
2451199523,35cyclictest1662-21chrt22:05:253
2451199522,36cyclictest11959-21chrt20:10:033
2451199522,34cyclictest29926-21chrt19:25:043
2451199514,34cyclictest24011-21taskset20:45:353
2451199513,38cyclictest5471-21taskset22:10:403
2451199513,38cyclictest27869-21chrt23:40:293
2451199513,37cyclictest31044-21chrt23:45:363
2451199512,38cyclictest15197-21taskset20:20:043
2451199512,35cyclictest16948-21taskset21:38:383
2451199503,35cyclictest23761-21taskset00:25:383
2451199503,35cyclictest23761-21taskset00:25:383
2451199503,33cyclictest4011-21chrt23:55:163
2451199502,37cyclictest32107-21chrt21:10:253
2451199502,37cyclictest32107-21chrt21:10:253
2451199502,36cyclictest27605-21chrt21:55:163
2451199502,36cyclictest12265-21chrt21:30:313
2451199502,35cyclictest16832-21chrt23:22:183
2451199502,34cyclictest27439-21chrt19:15:373
2451199502,34cyclictest12907-21chrt20:10:403
2451199495,34cyclictest23731-21taskset21:49:363
2451199493,36cyclictest10610-21chrt21:28:083
2451199493,33cyclictest30628-21chrt22:00:103
2451199493,33cyclictest19277-21chrt22:33:573
2451199493,32cyclictest32746-21chrt19:32:583
2451199493,32cyclictest32746-21chrt19:32:583
2451199493,32cyclictest32746-21chrt19:32:583
2451199493,32cyclictest28666-21chrt00:35:023
2451199493,32cyclictest22823-21chrt22:40:013
2451199493,31cyclictest5869-21chrt19:50:233
2451199492,34cyclictest2942-21taskset19:40:313
225552490,8sleep30-21swapper/320:40:373
225552490,8sleep30-21swapper/320:40:373
2451199484,31cyclictest14049-21chrt23:18:053
2451199483,35cyclictest23614-21chrt23:34:133
2451199483,35cyclictest23614-21chrt23:34:133
2451199483,33cyclictest20560-21chrt00:20:363
2451199482,33cyclictest10161-21chrt23:10:483
2451199474,30cyclictest9012-21taskset20:00:193
2451199473,33cyclictest5003-21chrt21:19:413
2451199473,33cyclictest5003-21chrt21:19:413
2451199453,29cyclictest26750-21chrt22:45:283
2451199443,30cyclictest3223-21taskset23:00:233
2451199434,30cyclictest20380-21taskset20:35:263
2451199432,27cyclictest15618-21chrt20:20:243
2451199424,29cyclictest26349-21taskset23:38:233
2451199393,26cyclictest26846-21chrt20:55:193
2451199393,26cyclictest19030-21chrt20:30:283
2451199393,24cyclictest16871-21taskset20:25:063
2451199383,27cyclictest24696-21chrt19:10:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional