You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-22 - 07:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Tue Sep 22, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25531212112,70sleep024596-21/usr/sbin/munin19:04:300
26096991192,27cyclictest22920-21proc_pri21:44:160
26096991192,27cyclictest22920-21proc_pri21:44:160
896921100,10sleep08954-21apt-get23:08:550
260969911052,9cyclictest4595-21kworker/u17:1+rpciod22:44:190
260969911052,9cyclictest4595-21kworker/u17:1+rpciod22:44:190
26096991098,33cyclictest29996-21apt-get23:43:580
260969910810,79cyclictest18693-21sh23:24:450
261139910668,9cyclictest181rcu_preempt22:14:102
2612099963,74cyclictest19361-21chrt00:19:153
26096999660,13cyclictest26525-21kworker/u17:1+xprtiod23:50:110
26096999638,17cyclictest16679-21ssh23:20:550
2496729420,40sleep10-21swapper/119:04:141
26113999269,16cyclictest5503-21latency_hist19:43:472
26096999260,23cyclictest6935-21gpgv19:48:480
26113999175,10cyclictest16020-21unixbench_multi23:19:202
2609699896,29cyclictest18663-21apt-get00:19:020
26096998959,22cyclictest30057-21kworker/u18:1-events_unbound19:13:480
26096998761,17cyclictest777-21diskmemload21:24:190
26104998673,8cyclictest0-21swapper/123:58:441
26113998546,16cyclictest8457-21ssh21:19:512
26113998539,11cyclictest0-21swapper/221:52:212
26104998567,12cyclictest365-21lldpd23:29:061
26104998567,12cyclictest365-21lldpd23:29:061
26096998554,22cyclictest3203-21fschecks_time22:59:040
26113998458,7cyclictest181rcu_preempt20:44:012
26113998453,7cyclictest181rcu_preempt22:09:092
26096998466,12cyclictest26082-21cyclictest22:34:200
26113998342,13cyclictest25419-21ssh00:29:062
26113998333,33cyclictest39-21ksoftirqd/220:28:442
26096998356,18cyclictest326-21systemd-logind19:58:450
26113998267,9cyclictest9376-21kernelversion23:09:082
26113998246,13cyclictest181rcu_preempt23:40:502
26113998242,11cyclictest0-21swapper/223:47:212
2611399822,45cyclictest23348-21users21:44:222
2611399822,45cyclictest23348-21users21:44:222
2609699822,59cyclictest17244-21apt-get22:29:000
2609699822,59cyclictest17244-21apt-get22:29:000
26113998158,8cyclictest181rcu_preempt22:48:432
26113998158,8cyclictest181rcu_preempt22:48:432
26113998157,10cyclictest181rcu_preempt22:39:162
26113998153,11cyclictest181rcu_preempt21:59:122
26113998145,8cyclictest0-21swapper/220:29:012
26104998165,10cyclictest1326-21processes19:29:171
26104998164,11cyclictest359-21snmpd20:38:461
26104998164,11cyclictest359-21snmpd20:38:461
26104998160,15cyclictest17672-21open_inodes20:19:151
2609699812,42cyclictest18941-21cat00:18:460
2609699812,42cyclictest18941-21cat00:18:460
26113998059,15cyclictest200-21systemd-journal19:58:462
26113998057,8cyclictest181rcu_preempt20:38:352
26113998057,8cyclictest181rcu_preempt20:38:352
26113998055,8cyclictest181rcu_preempt23:28:492
26113998055,8cyclictest181rcu_preempt23:28:492
26113998053,11cyclictest29281-21ssh22:48:512
26113998039,12cyclictest7023-21/usr/sbin/munin23:59:222
26113998038,8cyclictest0-21swapper/223:24:422
26113998032,12cyclictest181rcu_preempt00:38:452
26113998032,12cyclictest181rcu_preempt00:38:452
26104998060,14cyclictest28381-21sh00:34:031
26104998060,14cyclictest28381-21sh00:34:031
2609699806,63cyclictest19210-21/usr/sbin/munin21:39:110
26096998040,22cyclictest14626-21ssh21:30:140
2515228012,39sleep30-21swapper/319:04:203
26113997958,8cyclictest181rcu_preempt22:19:192
26113997945,12cyclictest181rcu_preempt21:39:202
26104997958,14cyclictest8821-21sh23:08:471
26104997958,14cyclictest8821-21sh23:08:471
2610499793,44cyclictest31-21ksoftirqd/123:09:411
26096997960,13cyclictest15-21ksoftirqd/020:43:460
26113997856,16cyclictest26082-21cyclictest20:03:192
26113997856,16cyclictest26082-21cyclictest20:03:192
26113997853,11cyclictest181rcu_preempt19:14:172
26113997849,10cyclictest181rcu_preempt00:14:202
26113997849,10cyclictest181rcu_preempt00:14:202
26113997834,14cyclictest0-21swapper/220:07:372
26113997831,12cyclictest181rcu_preempt21:38:152
26113997831,12cyclictest181rcu_preempt21:38:152
26113997828,17cyclictest0-21swapper/219:34:012
26104997857,15cyclictest662-21cron19:18:441
2609699781,35cyclictest2624-21sendmail-msp22:58:430
2609699781,35cyclictest2624-21sendmail-msp22:58:430
26113997744,8cyclictest181rcu_preempt23:54:542
26113997742,10cyclictest181rcu_preempt21:15:202
2611399772,65cyclictest19237-21apt-get00:19:032
26113997711,26cyclictest39-21ksoftirqd/219:24:212
26113997711,26cyclictest39-21ksoftirqd/219:24:212
26104997742,26cyclictest26492-21kworker/u18:1+nfsiod23:02:071
26104997738,29cyclictest26042-21kworker/u18:1+nfsiod23:45:151
2610499773,38cyclictest31-21ksoftirqd/122:57:371
2610499773,38cyclictest31-21ksoftirqd/122:57:371
26096997753,14cyclictest28273-21diskmemload21:53:540
26096997753,14cyclictest28273-21diskmemload21:53:540
2609699774,35cyclictest343-21runrttasks21:19:460
2609699772,38cyclictest13014-21sort00:08:550
26113997654,8cyclictest181rcu_preempt23:18:462
26113997654,8cyclictest181rcu_preempt23:18:462
26113997653,9cyclictest181rcu_preempt23:51:112
26113997653,9cyclictest181rcu_preempt21:04:002
26113997650,10cyclictest181rcu_preempt21:30:332
2611399762,45cyclictest39-21ksoftirqd/222:57:532
2611399762,45cyclictest39-21ksoftirqd/222:57:532
2611399762,33cyclictest3329-21irqstats22:59:072
2610499768,28cyclictest31-21ksoftirqd/100:24:541
2610499768,28cyclictest31-21ksoftirqd/100:24:541
26096997656,13cyclictest4595-21kworker/u17:1+xprtiod22:39:020
26096997646,21cyclictest0-21swapper/000:29:230
2609699764,57cyclictest5009-21cpuspeed22:09:020
26096997621,39cyclictest30155-21switchtime22:49:180
138042762,7sleep00-21swapper/022:23:470
26113997542,8cyclictest181rcu_preempt21:23:532
26113997540,11cyclictest181rcu_preempt19:19:042
26113997540,11cyclictest181rcu_preempt19:19:042
26113997535,10cyclictest181rcu_preempt19:29:112
26113997528,16cyclictest0-21swapper/200:27:452
26113997528,16cyclictest0-21swapper/200:27:452
26104997562,9cyclictest26082-21cyclictest21:03:451
26104997538,10cyclictest0-21swapper/120:39:061
26104997534,10cyclictest0-21swapper/123:14:011
26104997534,10cyclictest0-21swapper/123:14:011
26096997545,23cyclictest3990-21/usr/sbin/munin23:54:160
26096997536,9cyclictest181rcu_preempt21:34:000
26096997536,9cyclictest181rcu_preempt21:34:000
2609699753,35cyclictest343-21runrttasks00:02:100
262042740,14sleep0211rcuc/023:36:400
26113997450,8cyclictest181rcu_preempt20:09:012
26113997438,9cyclictest0-21swapper/200:04:122
26113997437,10cyclictest26082-21cyclictest23:03:572
26113997437,10cyclictest26082-21cyclictest23:03:572
26113997435,10cyclictest181rcu_preempt23:34:002
26113997414,27cyclictest2510-21ntp_states22:04:132
26104997456,10cyclictest0-21swapper/122:13:471
26104997453,15cyclictest22332-21ssh21:44:011
26104997453,15cyclictest22332-21ssh21:44:011
26104997444,23cyclictest6476-21kworker/u18:0+nfsiod23:49:281
2610499742,41cyclictest31-21ksoftirqd/122:02:011
26096997454,15cyclictest3904-21latency_hist19:38:470
26096997450,18cyclictest12990-21switchtime20:04:190
2609699743,55cyclictest438-21modprobe19:28:430
2609699743,55cyclictest438-21modprobe19:28:430
26113997355,12cyclictest7578-21cstates19:49:532
2611399735,29cyclictest14608-21ntp_states22:24:152
26113997350,8cyclictest181rcu_preempt19:13:472
26113997337,8cyclictest181rcu_preempt19:44:002
26113997333,16cyclictest181rcu_preempt22:34:422
26104997358,9cyclictest23328-21cpu22:39:001
26104997357,10cyclictest17875-21sh00:16:321
26104997357,10cyclictest17875-21sh00:16:321
26104997346,8cyclictest181rcu_preempt00:05:391
26104997335,11cyclictest343-21runrttasks00:33:151
2610499733,47cyclictest18255-21apt-get23:24:011
26104997313,53cyclictest26411-21kworker/u17:2+flush-179:022:29:191
26104997313,53cyclictest26411-21kworker/u17:2+flush-179:022:29:191
2609699733,61cyclictest18955-21apt-get20:23:570
2609699733,40cyclictest343-21runrttasks22:14:040
2609699733,35cyclictest4886-21awk21:14:170
253772737,42sleep225442-21cat19:04:282
26113997250,10cyclictest181rcu_preempt21:00:572
26113997241,9cyclictest181rcu_preempt20:19:162
26113997227,10cyclictest0-21swapper/221:53:472
26113997227,10cyclictest0-21swapper/221:53:472
26104997255,11cyclictest1-21systemd20:03:441
26104997255,11cyclictest1-21systemd20:03:441
26104997250,16cyclictest13262-21cpu21:29:051
26096997245,10cyclictest181rcu_preempt23:03:580
26096997245,10cyclictest181rcu_preempt23:03:580
2609699723,52cyclictest27230-21unixbench_singl19:09:220
26096997235,24cyclictest58-21kcompactd021:12:100
26096997235,24cyclictest58-21kcompactd021:12:100
2609699723,39cyclictest27788-21latency23:39:080
26113997148,8cyclictest181rcu_preempt00:08:592
26113997143,11cyclictest181rcu_preempt20:14:052
26113997132,10cyclictest58-21kcompactd021:09:242
26113997132,10cyclictest58-21kcompactd021:09:242
26113997129,14cyclictest181rcu_preempt22:31:472
26113997129,14cyclictest181rcu_preempt22:31:472
2610499715,24cyclictest31-21ksoftirqd/122:44:441
2610499715,24cyclictest31-21ksoftirqd/122:44:441
2610499714,45cyclictest20404-21latency_hist20:28:461
26096997141,13cyclictest181rcu_preempt21:48:520
2609699712,37cyclictest12728-21runrttasks00:08:440
26120997033,26cyclictest4036-21chrt23:53:573
2611399703,47cyclictest24186-21cat20:39:092
2610499709,33cyclictest31-21ksoftirqd/122:06:021
26104997051,12cyclictest16703-21gpgv22:28:471
26104997051,12cyclictest0-21swapper/120:13:461
26104997049,12cyclictest20577-21cpu20:29:011
26104997045,17cyclictest28480-21latency_hist20:53:461
26104997044,20cyclictest777-21diskmemload22:19:041
2610499703,44cyclictest30059-21latency_hist20:58:461
2610499703,28cyclictest31-21ksoftirqd/119:24:041
2610499703,28cyclictest31-21ksoftirqd/119:24:041
2610499702,32cyclictest31-21ksoftirqd/121:42:111
26096997030,10cyclictest181rcu_preempt20:48:560
26096997024,24cyclictest22499-21fschecks_time00:24:060
26096997024,24cyclictest22499-21fschecks_time00:24:060
2610499696,24cyclictest31-21ksoftirqd/119:43:461
26104996953,10cyclictest777-21diskmemload22:34:011
2610499694,39cyclictest31-21ksoftirqd/121:35:301
2610499694,39cyclictest31-21ksoftirqd/121:35:301
2610499692,36cyclictest31-21ksoftirqd/120:14:411
2610499692,27cyclictest31-21ksoftirqd/120:48:061
26104996911,22cyclictest31-21ksoftirqd/121:54:531
26104996911,22cyclictest31-21ksoftirqd/121:54:531
26096996948,11cyclictest0-21swapper/021:03:450
2609699693,36cyclictest14296-21fschecks_time22:24:060
26096996930,14cyclictest20835-21fschecks_time20:29:080
2609699692,45cyclictest16056-21wc20:14:160
2609699692,36cyclictest10908-21cut19:59:090
2609699692,36cyclictest10908-21cut19:59:090
2612099686,44cyclictest0-21swapper/320:25:023
26120996811,37cyclictest18163-21taskset20:20:073
2610499686,24cyclictest31-21ksoftirqd/123:34:201
26104996846,8cyclictest181rcu_preempt21:53:191
2610499684,43cyclictest2529-21sort19:34:061
2610499684,42cyclictest343-21runrttasks21:23:261
26096996845,14cyclictest4687-21latency_hist22:08:460
26096996834,13cyclictest181rcu_preempt00:35:000
26096996834,13cyclictest181rcu_preempt00:35:000
26120996712,32cyclictest0-21swapper/322:29:193
26120996712,32cyclictest0-21swapper/322:29:193
2611399674,36cyclictest28610-21/usr/sbin/munin20:54:182
26104996748,13cyclictest19595-21fschecks_count00:19:301
2610499674,27cyclictest31-21ksoftirqd/100:09:001
26104996737,7cyclictest181rcu_preempt19:21:291
26104996737,7cyclictest181rcu_preempt19:21:291
2610499672,54cyclictest7463-21sshd19:49:421
2610499672,28cyclictest31-21ksoftirqd/100:01:411
26096996746,12cyclictest0-21swapper/019:43:440
26104996647,13cyclictest777-21diskmemload23:42:531
2610499664,42cyclictest7157-21sed19:48:471
2610499663,27cyclictest31-21ksoftirqd/123:21:251
26104996620,34cyclictest10401-21sshd19:58:441
26104996617,9cyclictest291rcuc/122:14:101
2609699664,33cyclictest21240-21/usr/sbin/munin23:29:170
2609699664,33cyclictest21240-21/usr/sbin/munin23:29:170
2609699663,30cyclictest343-21runrttasks23:18:170
2609699663,30cyclictest343-21runrttasks23:18:170
26096996632,8cyclictest181rcu_preempt20:19:010
30702650,7sleep30-21swapper/322:04:323
26113996526,10cyclictest0-21swapper/220:53:062
2610499653,40cyclictest32723-21timerwakeupswit21:04:191
26104996529,20cyclictest31-21ksoftirqd/121:09:011
26104996529,20cyclictest31-21ksoftirqd/121:09:011
2610499652,30cyclictest31-21ksoftirqd/121:14:451
2609699653,34cyclictest14366-21ntp_kernel_pll_20:09:120
2609699653,27cyclictest23515-21cron20:38:430
2609699653,27cyclictest23515-21cron20:38:430
26096996518,34cyclictest31627-21fschecks_time21:59:070
26104996435,20cyclictest30358-21unixbench_singl22:49:201
26096996447,11cyclictest326-21systemd-logind20:48:440
2609699644,39cyclictest32747-21timerwakeupswit21:04:190
2609699644,35cyclictest1078-21uname19:29:100
26096996421,12cyclictest181rcu_preempt19:19:000
26096996421,12cyclictest181rcu_preempt19:19:000
2612099634,40cyclictest20143-21taskset21:39:193
2610499634,24cyclictest31-21ksoftirqd/119:08:561
26104996339,10cyclictest181rcu_preempt21:24:031
2609699633,48cyclictest8742-21modprobe19:53:430
2612099623,45cyclictest9673-21taskset22:16:073
2612099623,36cyclictest25344-21chrt22:42:003
26120996212,37cyclictest0-21swapper/319:18:443
26120996112,30cyclictest0-21swapper/323:23:583
26120996111,37cyclictest15726-21taskset22:26:083
26096996122,30cyclictest29981-21idleruntime-cro20:58:430
2612099605,41cyclictest0-21swapper/323:44:133
2612099604,42cyclictest15950-21chrt23:19:203
2610499602,28cyclictest31-21ksoftirqd/120:04:021
2612099594,41cyclictest7716-21taskset23:06:113
2612099594,41cyclictest7716-21taskset23:06:113
2612099593,43cyclictest12189-21chrt23:14:053
2612099593,43cyclictest12189-21chrt23:14:053
2612099585,34cyclictest15807-21chrt20:14:083
2612099583,38cyclictest32386-21chrt21:59:213
26120995819,24cyclictest18799-21taskset21:38:303
26120995819,24cyclictest18799-21taskset21:38:303
2612099573,39cyclictest30114-21chrt22:49:183
2612099573,39cyclictest21340-21taskset23:29:003
2612099573,39cyclictest21340-21taskset23:29:003
26120995719,23cyclictest1790-21taskset23:49:193
2612099564,38cyclictest30425-21chrt23:43:483
2612099564,38cyclictest22515-21taskset21:44:063
2612099564,38cyclictest22515-21taskset21:44:063
2612099562,40cyclictest22352-21chrt00:24:013
2612099562,40cyclictest22352-21chrt00:24:013
26120995622,23cyclictest26616-21taskset22:44:073
26120995622,23cyclictest26616-21taskset22:44:073
26120995619,23cyclictest3021-21chrt21:11:353
26120995619,23cyclictest3021-21chrt21:11:353
26120995616,27cyclictest13034-21taskset00:08:573
2612099553,39cyclictest1615-21chrt19:29:213
26120995518,22cyclictest24083-21chrt20:39:073
26120995517,25cyclictest12713-21taskset20:04:123
26120995516,24cyclictest776-21chrt21:08:433
26120995515,26cyclictest17473-21taskset00:15:263
26120995515,26cyclictest17473-21taskset00:15:263
2612099544,37cyclictest11222-21chrt21:24:203
2612099544,35cyclictest11680-21chrt22:19:153
2612099543,37cyclictest30150-21chrt19:19:043
2612099543,37cyclictest30150-21chrt19:19:043
2612099543,37cyclictest26815-21chrt19:09:163
2612099543,35cyclictest15225-21taskset20:13:433
2612099543,34cyclictest5456-21chrt19:43:473
26120995422,20cyclictest9878-21chrt21:23:433
26120995422,20cyclictest25982-21chrt21:49:173
26120995421,21cyclictest31567-21chrt19:23:583
26120995421,21cyclictest31567-21chrt19:23:583
26120995420,22cyclictest2336-21chrt19:33:593
26120995419,23cyclictest20264-21taskset22:33:583
26120995419,22cyclictest6120-21chrt19:44:153
26120995418,22cyclictest11191-21chrt19:59:183
26120995418,22cyclictest11191-21chrt19:59:183
26120995416,24cyclictest28536-21taskset00:34:073
26120995416,24cyclictest28536-21taskset00:34:073
26120995411,29cyclictest4976-21chrt21:14:183
2612099533,36cyclictest27865-21chrt00:33:423
2612099533,34cyclictest8816-21chrt00:01:153
2612099533,33cyclictest13462-21chrt21:29:103
2612099532,38cyclictest2068-21chrt22:57:203
2612099532,38cyclictest2068-21chrt22:57:203
26120995321,20cyclictest29887-21chrt21:55:453
26120995321,20cyclictest29887-21chrt21:55:453
26120995320,20cyclictest9470-21chrt19:54:113
26120995318,21cyclictest29286-21taskset20:54:173
26120995317,24cyclictest25920-21taskset23:35:543
26120995317,22cyclictest9916-21taskset23:09:193
2612099524,24cyclictest0-21swapper/319:48:583
2612099523,35cyclictest27356-21chrt20:49:073
26120995219,21cyclictest6018-21chrt22:09:223
2612099514,35cyclictest4667-21chrt23:01:043
2612099514,32cyclictest20708-21taskset20:29:023
2612099513,36cyclictest30523-21taskset20:59:073
26120995118,21cyclictest10034-21chrt00:03:543
26120995117,21cyclictest22230-21chrt20:34:003
26120995117,21cyclictest22230-21chrt20:34:003
26120995016,22cyclictest25753-21chrt20:44:093
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional