You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-27 - 19:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sat Dec 27, 2025 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23058214312,101sleep10-21swapper/107:05:241
2322121269,85sleep023276-21timerandwakeup07:05:270
2620421160,7sleep126203-21diskstats10:49:591
239409911052,12cyclictest18714-21ssh09:42:200
239409911052,12cyclictest18714-21ssh09:42:200
239579910184,11cyclictest31322-21diskmemload10:15:162
2394099996,62cyclictest31519-21runrttasks07:30:090
23957999779,12cyclictest346-21runrttasks08:34:422
23949999269,16cyclictest20264-21fschecks_time12:29:591
23957999150,8cyclictest181rcu_preempt09:27:132
23957999150,8cyclictest181rcu_preempt09:27:132
23940998865,15cyclictest15-21ksoftirqd/010:29:420
23940998854,11cyclictest181rcu_preempt10:49:540
23940998851,29cyclictest15-21ksoftirqd/010:04:580
23940998749,10cyclictest181rcu_preempt09:29:530
2394099873,56cyclictest26426-21/usr/sbin/munin07:15:110
23949998664,12cyclictest0-21swapper/107:50:161
23940998656,23cyclictest22185-21kworker/u18:0+events_unbound08:59:390
23940998656,23cyclictest22185-21kworker/u18:0+events_unbound08:59:390
23940998651,28cyclictest1-21systemd11:39:390
23940998648,10cyclictest181rcu_preempt08:09:560
23940998648,10cyclictest181rcu_preempt08:09:560
23940998642,34cyclictest0-21swapper/011:04:410
23940998639,37cyclictest0-21swapper/011:40:030
23957998565,14cyclictest23927-21cyclictest09:06:092
23957998549,24cyclictest13688-21systemctl10:29:392
23949998565,15cyclictest4854-21sh09:19:421
23949998564,9cyclictest0-21swapper/107:14:521
23957998468,8cyclictest0-21swapper/210:20:572
23957998466,13cyclictest0-21swapper/211:10:082
23957998466,13cyclictest0-21swapper/211:10:082
23940998451,12cyclictest181rcu_preempt10:34:040
23940998451,12cyclictest181rcu_preempt10:34:040
23957998359,18cyclictest1-21systemd07:50:082
23957998356,7cyclictest181rcu_preempt12:38:302
23949998367,9cyclictest0-21swapper/109:54:571
23949998361,13cyclictest0-21swapper/107:29:411
23949998353,23cyclictest1286-21kworker/u18:1+nfsiod10:19:591
23957998260,12cyclictest0-21swapper/212:04:392
23957998252,9cyclictest181rcu_preempt10:35:292
23957998251,14cyclictest0-21swapper/210:46:242
23949998260,12cyclictest0-21swapper/107:35:101
23940998259,14cyclictest29511-21ntp_kernel_err10:55:080
23940998259,14cyclictest29511-21ntp_kernel_err10:55:080
23940998258,15cyclictest26242-21fschecks_count11:45:030
23940998258,15cyclictest26242-21fschecks_count11:45:030
23940998251,24cyclictest0-21swapper/007:45:170
2394099824,52cyclictest25467-21munin-run07:14:390
23940998237,20cyclictest181rcu_preempt12:05:060
23957998143,11cyclictest7455-21ssh09:23:522
23949998150,24cyclictest18196-21kworker/u18:0+nfsiod10:12:541
2394099813,62cyclictest23043-21apt-get10:44:530
23957998062,13cyclictest28497-21latency09:00:042
23957998059,10cyclictest0-21swapper/208:50:142
23957998058,14cyclictest2-21kthreadd09:39:412
23949998061,9cyclictest0-21swapper/110:49:411
23949998058,16cyclictest9217-21timerwakeupswit12:10:141
23940998061,11cyclictest23927-21cyclictest08:35:090
23940998048,8cyclictest181rcu_preempt08:17:170
23957997960,9cyclictest0-21swapper/207:14:402
23957997946,8cyclictest181rcu_preempt11:45:152
23957997946,8cyclictest181rcu_preempt11:45:152
23949997959,10cyclictest0-21swapper/111:30:101
23949997959,10cyclictest0-21swapper/111:30:101
23940997946,9cyclictest181rcu_preempt11:14:570
23940997944,11cyclictest181rcu_preempt09:59:510
23940997932,11cyclictest181rcu_preempt09:24:390
23957997859,10cyclictest0-21swapper/209:15:012
23957997858,15cyclictest22513-21sh09:49:412
23957997840,10cyclictest1353-21ssh10:08:172
2395799783,35cyclictest39-21ksoftirqd/208:57:332
2395799783,35cyclictest39-21ksoftirqd/208:57:332
23949997858,10cyclictest0-21swapper/112:15:101
23949997855,17cyclictest31322-21diskmemload10:04:181
23940997857,15cyclictest2159-21latency_hist11:59:410
23940997855,7cyclictest181rcu_preempt11:24:530
23940997855,7cyclictest181rcu_preempt11:24:530
23940997852,18cyclictest25667-21latency_hist12:39:410
23957997759,11cyclictest12660-21ssh12:16:232
23957997751,11cyclictest181rcu_preempt12:09:372
23957997748,9cyclictest181rcu_preempt11:41:532
23957997735,10cyclictest181rcu_preempt10:57:222
23957997735,10cyclictest181rcu_preempt10:57:222
23957997726,33cyclictest500-21grep11:00:172
23949997755,12cyclictest0-21swapper/109:39:591
23949997755,12cyclictest0-21swapper/109:39:591
23949997750,9cyclictest181rcu_preempt11:05:101
23949997745,23cyclictest14898-21modprobe08:19:381
23940997755,16cyclictest32272-21sh09:10:110
23940997751,11cyclictest17537-21ssh10:35:100
23940997748,22cyclictest0-21swapper/010:09:560
2394099773,35cyclictest823-21fschecks_count07:35:000
2394099772,23cyclictest23351-21ls08:44:560
23957997662,10cyclictest23927-21cyclictest09:13:312
23957997653,16cyclictest17473-21fschecks_time11:30:012
23957997653,16cyclictest17473-21fschecks_time11:30:012
23957997649,18cyclictest29385-21latency11:50:032
23957997647,8cyclictest181rcu_preempt11:38:092
2395799762,41cyclictest346-21runrttasks07:15:112
23949997659,9cyclictest0-21swapper/111:14:411
23949997659,9cyclictest0-21swapper/111:14:411
23949997659,10cyclictest0-21swapper/111:55:051
23940997658,12cyclictest358-21ntpd12:30:010
23940997621,9cyclictest181rcu_preempt11:50:130
23957997556,10cyclictest0-21swapper/208:20:012
23957997553,16cyclictest26243-21cut10:50:002
23957997552,8cyclictest181rcu_preempt11:05:062
23957997550,8cyclictest181rcu_preempt07:25:072
23957997545,22cyclictest2027-21latency_hist11:59:402
23957997541,18cyclictest0-21swapper/212:20:542
23949997559,8cyclictest0-21swapper/111:35:141
23949997558,10cyclictest0-21swapper/109:00:111
23949997555,14cyclictest23852-21unixbench_singl09:50:161
23940997558,11cyclictest20025-21ssh10:39:560
23940997538,10cyclictest181rcu_preempt09:45:070
23957997460,8cyclictest0-21swapper/208:25:132
23957997458,11cyclictest23640-21/usr/sbin/munin09:50:142
23957997454,7cyclictest181rcu_preempt08:05:022
23957997454,7cyclictest181rcu_preempt08:05:022
23957997446,7cyclictest181rcu_preempt10:30:322
23957997446,7cyclictest181rcu_preempt10:30:322
23949997458,12cyclictest23927-21cyclictest10:15:201
23949997453,15cyclictest1-21systemd08:24:391
23940997451,15cyclictest201-21systemd-journal11:21:070
23940997449,11cyclictest181rcu_preempt09:54:530
23940997410,32cyclictest29459-21apt-get09:04:380
23957997352,12cyclictest329-21systemd-logind11:24:402
23957997350,10cyclictest181rcu_preempt08:38:232
23957997347,9cyclictest181rcu_preempt10:40:132
23957997344,7cyclictest181rcu_preempt11:18:382
23957997342,10cyclictest181rcu_preempt10:01:212
23957997338,12cyclictest181rcu_preempt12:33:532
23949997358,7cyclictest0-21swapper/111:19:571
23949997354,9cyclictest0-21swapper/108:55:131
23949997354,9cyclictest0-21swapper/108:55:131
23940997340,14cyclictest0-21swapper/007:50:070
23940997340,13cyclictest181rcu_preempt11:30:150
23940997340,13cyclictest181rcu_preempt11:30:150
23957997257,9cyclictest0-21swapper/207:44:582
23957997251,15cyclictest23845-21processes08:45:112
23957997247,6cyclictest181rcu_preempt12:26:332
23957997240,8cyclictest181rcu_preempt11:29:122
23957997240,8cyclictest181rcu_preempt11:29:122
23957997237,7cyclictest181rcu_preempt12:09:542
2395799722,30cyclictest39-21ksoftirqd/210:13:582
23949997249,18cyclictest5945-21kworker/u19:0+events_unbound09:14:391
2394099725,44cyclictest19685-21idleruntime-cro12:29:380
23940997250,14cyclictest60-21kcompactd012:14:380
23940997248,15cyclictest23532-21ssh09:50:120
23940997245,9cyclictest23927-21cyclictest12:22:240
23957997150,15cyclictest380-21in:imuxsock07:32:072
23957997141,15cyclictest181rcu_preempt09:31:472
23957997137,9cyclictest181rcu_preempt07:34:572
23949997153,11cyclictest25495-21systemd-run07:14:401
23940997146,16cyclictest0-21swapper/009:25:160
23940997146,16cyclictest0-21swapper/009:25:160
23940997132,29cyclictest0-21swapper/012:00:060
2394099712,35cyclictest2323-21df_inode09:14:580
23957997053,11cyclictest17226-21fschecks_time09:40:042
23957997053,11cyclictest17226-21fschecks_time09:40:042
2395799703,27cyclictest39-21ksoftirqd/209:54:552
23949997056,8cyclictest32710-21/usr/sbin/munin11:00:151
23949997053,7cyclictest0-21swapper/108:50:131
23949997027,31cyclictest17019-21cpuspeed10:34:561
23940997055,10cyclictest23927-21cyclictest11:08:060
23940997054,10cyclictest3253-21unixbench_singl07:40:160
23940997047,16cyclictest16733-21latency_hist09:39:410
2394099703,51cyclictest8060-21head07:55:150
2394099703,35cyclictest31329-21taskset07:29:430
2383627010,37sleep223854-21unin-run07:09:412
23957996952,10cyclictest0-21swapper/207:55:142
2395799692,28cyclictest39-21ksoftirqd/207:19:542
23949996946,17cyclictest21830-21fschecks_count08:40:001
23949996944,19cyclictest2470-21fschecks_count07:40:001
23949996937,7cyclictest181rcu_preempt12:05:171
23949996935,9cyclictest23927-21cyclictest08:29:091
2394099695,32cyclictest11288-21/usr/sbin/munin12:14:560
23940996940,21cyclictest11101-21threads08:05:150
23940996940,21cyclictest11101-21threads08:05:150
23940996928,13cyclictest181rcu_preempt08:24:530
2394099692,26cyclictest25085-21fschecks_count08:50:000
23957996849,12cyclictest3657-21systemd07:44:412
23957996838,8cyclictest181rcu_preempt08:15:092
23957996834,8cyclictest181rcu_preempt07:59:572
23957996834,8cyclictest181rcu_preempt07:59:572
23949996853,7cyclictest0-21swapper/111:40:121
23949996852,9cyclictest0-21swapper/110:44:391
23949996845,18cyclictest23927-21cyclictest09:35:101
2394999682,28cyclictest31-21ksoftirqd/110:24:581
23940996849,13cyclictest161ktimers/011:09:520
23940996849,13cyclictest161ktimers/011:09:520
23940996843,15cyclictest0-21swapper/010:15:180
23940996838,8cyclictest0-21swapper/010:21:280
23940996814,10cyclictest30260-21fschecks_count09:05:560
2396499673,45cyclictest28444-21taskset09:59:413
23949996751,10cyclictest7078-21/usr/sbin/munin07:55:061
23949996748,13cyclictest14965-21mailstats11:25:151
23949996748,13cyclictest14965-21mailstats11:25:151
23949996743,17cyclictest363-21snmpd11:47:031
23949996743,17cyclictest363-21snmpd11:47:031
23949996730,7cyclictest181rcu_preempt08:35:081
2394999672,30cyclictest31-21ksoftirqd/108:29:501
2394999672,27cyclictest31-21ksoftirqd/108:45:051
2394099673,31cyclictest15805-21processes08:20:110
2396499662,53cyclictest0-21swapper/310:45:453
2394999665,36cyclictest32466-21unixbench_singl07:30:481
23949996651,11cyclictest23927-21cyclictest08:04:421
23949996641,17cyclictest23927-21cyclictest09:25:001
23949996641,17cyclictest23927-21cyclictest09:25:001
2394999663,38cyclictest29943-21/usr/sbin/munin09:06:081
23949996628,10cyclictest181rcu_preempt11:16:371
23949996610,41cyclictest2801-21kernelversion12:00:041
2395799653,43cyclictest13331-21latency_hist08:14:402
2395799653,43cyclictest13331-21latency_hist08:14:402
23949996548,9cyclictest0-21swapper/110:55:061
23949996548,9cyclictest0-21swapper/110:55:061
23949996546,14cyclictest5945-21kworker/u19:0+events_unbound09:24:381
23949996543,13cyclictest0-21swapper/109:45:141
23949996536,7cyclictest181rcu_preempt12:39:091
23949996535,8cyclictest181rcu_preempt10:07:361
2394999652,27cyclictest31-21ksoftirqd/108:10:101
2394999652,27cyclictest31-21ksoftirqd/108:10:101
2394099654,39cyclictest28138-21cstates07:19:580
23957996448,9cyclictest0-21swapper/208:40:042
23949996441,14cyclictest0-21swapper/110:34:411
23949996441,14cyclictest0-21swapper/110:34:411
23949996441,13cyclictest0-21swapper/108:05:021
23949996441,13cyclictest0-21swapper/108:05:021
23949996431,7cyclictest181rcu_preempt12:28:381
23949996420,9cyclictest291rcuc/109:29:551
23949996414,36cyclictest31585-21wc11:54:401
23949996332,7cyclictest181rcu_preempt07:49:401
23949996330,8cyclictest181rcu_preempt12:24:111
23940996337,9cyclictest181rcu_preempt07:59:570
23940996337,9cyclictest181rcu_preempt07:59:570
2394099633,41cyclictest346-21runrttasks08:29:590
23964996223,26cyclictest26023-21taskset11:44:553
23964996223,26cyclictest26023-21taskset11:44:553
23964996221,28cyclictest3374-21taskset09:15:403
2394999623,42cyclictest8912-21fschecks_count08:00:011
23964996126,20cyclictest20302-21chrt10:40:043
23964996124,27cyclictest10159-21chrt08:04:403
23964996124,27cyclictest10159-21chrt08:04:403
23964996124,23cyclictest19847-21chrt08:34:413
23964996122,25cyclictest22312-21chrt11:38:353
23964996120,29cyclictest22804-21taskset09:49:523
2394099613,38cyclictest21844-21tune2fs08:40:000
2396499603,42cyclictest6733-21taskset10:16:503
23964996026,20cyclictest16795-21chrt08:24:493
23964996025,24cyclictest7594-21chrt09:24:153
23964996024,23cyclictest26311-21chrt10:50:013
23964996023,23cyclictest6788-21chrt12:06:223
2394999603,32cyclictest28216-21fschecks_count07:20:021
23964995925,23cyclictest23304-21chrt12:35:023
23964995924,22cyclictest7243-21chrt07:54:573
23964995924,22cyclictest31344-21chrt11:54:063
23964995923,24cyclictest19273-21taskset10:38:543
23964995921,26cyclictest21309-21taskset08:39:113
23964995920,27cyclictest1088-21taskset07:35:043
23964995919,28cyclictest25367-21taskset07:13:383
2396499583,40cyclictest24267-21taskset11:40:373
2396499583,38cyclictest15002-21chrt12:20:163
2396499583,36cyclictest2677-21taskset11:05:003
23964995825,22cyclictest12402-21taskset12:15:383
23964995824,21cyclictest13400-21chrt09:34:043
23964995823,24cyclictest31823-21taskset11:54:473
23964995823,23cyclictest10177-21chrt10:23:153
23964995821,25cyclictest2756-21taskset07:40:063
23964995820,25cyclictest8256-21chrt12:09:503
23964995820,24cyclictest23411-21taskset08:44:583
23964995726,20cyclictest4154-21chrt10:12:533
23964995726,19cyclictest2744-21chrt12:00:023
23964995724,21cyclictest4846-21chrt07:45:173
23964995724,21cyclictest31657-21chrt09:09:533
23964995724,21cyclictest19691-21chrt11:34:293
23964995724,21cyclictest19691-21chrt11:34:293
23964995724,20cyclictest8342-21chrt09:25:003
23964995724,20cyclictest8342-21chrt09:25:003
23964995723,22cyclictest15937-21taskset09:37:533
23964995723,21cyclictest9676-21chrt11:16:063
23964995722,21cyclictest26488-21chrt08:54:543
23964995722,21cyclictest26488-21chrt08:54:543
23964995722,21cyclictest1668-21chrt11:03:263
23964995721,25cyclictest19368-21taskset09:44:133
23964995721,25cyclictest19368-21taskset09:44:133
23964995721,24cyclictest32380-21taskset07:30:473
23964995721,24cyclictest28887-21chrt09:59:573
23964995721,23cyclictest31852-21taskset10:04:593
23964995720,24cyclictest21323-21taskset12:30:573
23964995624,21cyclictest13388-21chrt08:14:413
23964995624,21cyclictest13388-21chrt08:14:413
23964995622,23cyclictest13309-21chrt10:28:423
23964995621,24cyclictest7054-21taskset11:12:013
23964995621,24cyclictest7054-21taskset11:12:013
23964995621,24cyclictest28961-21chrt10:54:513
23964995621,24cyclictest28961-21chrt10:54:513
23964995621,21cyclictest15277-21chrt11:25:183
23964995621,21cyclictest15277-21chrt11:25:183
23964995524,21cyclictest26522-21chrt07:15:123
23964995523,20cyclictest31228-21chrt09:08:473
23964995519,23cyclictest15543-21taskset08:20:033
2396499543,36cyclictest11597-21taskset11:20:033
2396499542,39cyclictest10391-21taskset08:04:573
2396499542,39cyclictest10391-21taskset08:04:573
23964995421,20cyclictest6542-21chrt07:51:043
23964995420,23cyclictest21570-21chrt09:47:053
23964995414,28cyclictest29173-21taskset07:22:083
23964995414,28cyclictest22910-21taskset08:43:443
2396499534,32cyclictest26085-21taskset08:53:083
2396499533,38cyclictest28217-21taskset08:59:563
2396499533,36cyclictest18252-21taskset12:25:303
2396499532,33cyclictest14310-21chrt08:15:133
23964995314,25cyclictest14162-21taskset10:29:593
23964995314,25cyclictest14162-21taskset10:29:593
23964995221,20cyclictest29691-21chrt07:24:563
224782459,22sleep30-21swapper/307:05:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional