You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 16:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sat Aug 01, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
322421110,7sleep30-21swapper/309:06:243
322421110,7sleep30-21swapper/309:06:243
205569910718,37cyclictest9276-21cpuspeed10:11:062
205569910718,37cyclictest9276-21cpuspeed10:11:062
20545991052,79cyclictest3610-21cpuspeed07:42:090
205519910255,38cyclictest9277-21sort10:11:061
205519910255,38cyclictest9277-21sort10:11:061
205459910282,15cyclictest23609-21kworker/u18:1+events_unbound10:35:480
205459910282,15cyclictest23609-21kworker/u18:1+events_unbound10:35:480
20556991012,43cyclictest26870-21timerwakeupswit07:11:252
20556991012,43cyclictest26870-21timerwakeupswit07:11:252
2055699996,47cyclictest21475-21ssh10:31:052
2055699996,47cyclictest21475-21ssh10:31:052
2055199993,27cyclictest24790-21taskset12:24:541
20545999871,15cyclictest8641-21ssh09:16:150
2055699974,32cyclictest3135-21/usr/sbin/munin10:01:222
2055699974,32cyclictest3135-21/usr/sbin/munin10:01:222
2055199972,38cyclictest24122-21munin-run07:05:491
2055199972,38cyclictest24122-21munin-run07:05:491
20556999546,12cyclictest352-21runrttasks11:58:522
2055699953,40cyclictest5034-21latency_hist07:45:522
2055699944,36cyclictest14554-21/usr/sbin/munin09:26:172
2055699943,39cyclictest5114-21fschecks_count11:51:542
2055699943,39cyclictest5114-21fschecks_count11:51:542
2055699923,39cyclictest6734-21sh07:50:492
20551999275,11cyclictest10381-21latency_hist12:00:511
2055699913,38cyclictest1079-21timerandwakeup09:56:272
201072917,69sleep20-21swapper/206:52:042
20551999065,8cyclictest30652cat10:00:541
2055199903,34cyclictest14618-21unixbench_singl08:11:261
20545999074,10cyclictest4811-21apt-get07:46:050
2055699893,38cyclictest14323-21sensors_temp12:06:242
2055699893,36cyclictest29401-21cut07:21:112
2055199894,31cyclictest22730-21tail10:32:191
2055199894,31cyclictest22730-21tail10:32:191
2054599894,48cyclictest15-21ksoftirqd/009:05:510
2054599894,46cyclictest15-21ksoftirqd/007:01:200
2054599894,46cyclictest15-21ksoftirqd/007:01:200
2054599891,47cyclictest15-21ksoftirqd/009:11:160
20551998844,36cyclictest28433-21diskmemload10:16:191
2055199883,58cyclictest5363-21runrttasks07:46:151
2055199882,34cyclictest16720-21df12:11:101
2055199882,33cyclictest10868-21tune2fs12:01:121
20545998862,19cyclictest24856-21needreboot07:06:150
20545998862,19cyclictest24856-21needreboot07:06:150
2055699876,39cyclictest1617-21/usr/sbin/munin10:51:242
2055699876,39cyclictest1617-21/usr/sbin/munin10:51:242
2055699874,36cyclictest352-21runrttasks07:07:142
2055699874,36cyclictest352-21runrttasks07:07:142
2055199873,7cyclictest0-21swapper/111:21:011
20545998769,13cyclictest20532-21cyclictest09:10:490
20545998769,13cyclictest20532-21cyclictest09:10:490
20545998761,18cyclictest16781-21latency_hist08:20:520
2055699866,30cyclictest11528-21/usr/sbin/munin09:21:152
2055699862,38cyclictest3018-21ssh11:47:502
2055199864,6cyclictest0-21swapper/107:56:051
2055199863,30cyclictest23316-21latency_hist09:40:501
2055199862,7cyclictest0-21swapper/107:06:041
2055199862,7cyclictest0-21swapper/107:06:041
20545998662,15cyclictest300-21rs:main0
20545998662,15cyclictest300-21rs:main0
2054599863,44cyclictest15-21ksoftirqd/012:06:440
20545998634,13cyclictest181rcu_preempt10:54:590
20545998634,13cyclictest181rcu_preempt10:54:590
2055199854,33cyclictest18214-21apt-get10:26:011
2055199853,8cyclictest0-21swapper/112:06:011
2055199853,33cyclictest32286-21latency_hist07:30:511
2055199853,31cyclictest3441-21ssh09:06:281
2055199853,31cyclictest3441-21ssh09:06:281
2055199853,30cyclictest27589-21latency_hist11:35:521
2054599853,48cyclictest15-21ksoftirqd/008:46:190
20556998463,14cyclictest20532-21cyclictest09:39:052
2055699843,36cyclictest21730-21grep08:35:492
2055699843,36cyclictest21730-21grep08:35:492
20551998445,25cyclictest0-21swapper/108:26:161
2055199844,42cyclictest30941-21grep10:46:181
2055199843,34cyclictest28447-21/usr/sbin/munin10:41:281
2055199843,33cyclictest16883-21acpi08:20:591
2055199843,31cyclictest26265-21latency_hist09:45:531
2055199843,30cyclictest24535-21cut11:30:521
2055199843,30cyclictest24535-21cut11:30:521
2055199842,6cyclictest0-21swapper/107:16:061
2055199842,34cyclictest3343-21/usr/sbin/munin10:01:081
2055199842,34cyclictest3343-21/usr/sbin/munin10:01:081
2055199842,34cyclictest27854-21ssh11:36:081
2055199842,34cyclictest27854-21ssh11:36:081
2055199842,33cyclictest5888-21tail11:52:101
2055199842,33cyclictest5888-21tail11:52:101
20556998363,6cyclictest181rcu_preempt11:36:102
20556998363,6cyclictest181rcu_preempt11:36:102
2055699835,37cyclictest7999-21apt-get09:16:032
2055699835,36cyclictest12285-21fschecks_count08:06:102
2055699835,36cyclictest12285-21fschecks_count08:06:102
2055199833,7cyclictest0-21swapper/108:41:061
2055199833,36cyclictest13097-21ssh11:11:111
2055199833,32cyclictest32326-21/usr/sbin/munin09:01:181
2055199833,32cyclictest30794-21ssh11:40:531
2055199833,32cyclictest30794-21ssh11:40:531
2055199833,31cyclictest8213-21apt-config09:15:511
2055199833,31cyclictest29449-21/usr/sbin/munin07:21:121
2055199832,37cyclictest17562-21apt-get09:31:001
2055199832,34cyclictest15105-21/usr/sbin/munin09:26:161
2055199832,31cyclictest32628-21/usr/sbin/munin07:31:101
2054599834,52cyclictest15-21ksoftirqd/012:15:340
20545998324,35cyclictest15-21ksoftirqd/011:42:300
20545998324,35cyclictest15-21ksoftirqd/011:42:300
20545998310,22cyclictest15-21ksoftirqd/010:55:580
2055199824,31cyclictest3777-21grep07:42:171
2055199823,7cyclictest0-21swapper/110:21:051
2055199823,31cyclictest20051-21/usr/sbin/munin12:16:181
2055199823,30cyclictest7129-21/usr/sbin/munin11:01:111
2055199822,7cyclictest0-21swapper/108:06:041
2055199822,7cyclictest0-21swapper/108:06:041
2055199822,40cyclictest26389-21awk08:46:261
2055199822,35cyclictest2311-21/usr/sbin/munin11:46:251
2055199822,34cyclictest9371-21cat09:16:381
2054599823,49cyclictest15-21ksoftirqd/009:45:530
20545998225,38cyclictest15-21ksoftirqd/007:30:520
2054599822,48cyclictest15-21ksoftirqd/009:21:270
20556998162,13cyclictest15168-21sh10:21:022
2055199813,8cyclictest0-21swapper/109:46:031
2055199813,30cyclictest25982-21modprobe07:10:521
2055199813,30cyclictest25982-21modprobe07:10:521
2055199812,8cyclictest0-21swapper/108:31:011
2055199812,8cyclictest0-21swapper/108:31:011
2055199812,7cyclictest0-21swapper/110:51:021
2055199812,7cyclictest0-21swapper/110:51:021
2055199812,35cyclictest4425-21/usr/sbin/munin10:56:201
2055199812,34cyclictest6388-21df_inode10:06:111
2055199812,33cyclictest10852-21/usr/sbin/munin08:02:191
2055199812,32cyclictest22954-21/usr/sbin/munin08:37:221
2055199812,32cyclictest22954-21/usr/sbin/munin08:37:221
2055199812,30cyclictest27442-21runrttasks08:51:141
2055199812,30cyclictest27442-21runrttasks08:51:141
20545998164,10cyclictest23013-21kworker/u17:0+xprtiod10:25:520
20545998162,12cyclictest18388-21latency_hist11:20:520
20545998137,30cyclictest181rcu_preempt11:25:490
2054599813,50cyclictest15-21ksoftirqd/008:39:590
2054599813,50cyclictest15-21ksoftirqd/008:39:590
2054599812,49cyclictest15-21ksoftirqd/009:59:130
201572815,58sleep00-21swapper/006:52:430
20556998061,14cyclictest10000-21sh11:06:102
20556998058,14cyclictest28444-21timerwakeupswit07:16:262
2055199804,30cyclictest2982-21http07:40:491
2055199803,34cyclictest29620-21runrttasks09:51:001
2055199803,31cyclictest21049-21/usr/sbin/munin06:56:131
2055199803,31cyclictest12496-21ssh09:21:261
2055199803,31cyclictest10953-21sh11:06:361
2055199802,32cyclictest15739-21ssh11:16:001
2055199802,31cyclictest20669-21latency_hist06:55:541
2054599808,47cyclictest15-21ksoftirqd/006:56:130
20556997958,15cyclictest20532-21cyclictest08:45:522
2055199793,8cyclictest0-21swapper/107:51:031
2055199793,32cyclictest28959-21/usr/sbin/munin08:56:101
2055199793,31cyclictest15847-21/usr/sbin/munin08:16:231
2055199792,34cyclictest24381-21apt-get10:36:021
20545997953,10cyclictest181rcu_preempt11:56:110
20545997941,30cyclictest0-21swapper/011:06:170
2054599793,38cyclictest15-21ksoftirqd/011:26:050
2054599793,38cyclictest15-21ksoftirqd/011:26:050
20545997918,36cyclictest15-21ksoftirqd/008:15:020
20556997860,13cyclictest27122-21sh11:35:122
20545997843,27cyclictest0-21swapper/011:52:070
20545997843,27cyclictest0-21swapper/011:52:070
20545997828,22cyclictest15-21ksoftirqd/011:39:450
20545997828,22cyclictest15-21ksoftirqd/011:39:450
20545997826,32cyclictest15-21ksoftirqd/010:45:510
20545997812,44cyclictest15-21ksoftirqd/008:26:230
20556997752,16cyclictest21574-21latency_hist11:25:522
2054599778,37cyclictest13201-21munin-run12:05:480
20545997759,12cyclictest12453-21ssh10:16:110
20545997755,16cyclictest23013-21kworker/u17:0+nfsiod11:31:420
20545997728,31cyclictest15-21ksoftirqd/010:15:520
20545997728,31cyclictest15-21ksoftirqd/010:15:520
2054599772,29cyclictest17460-21sshd09:37:460
20556997653,13cyclictest0-21swapper/211:26:262
20556997653,13cyclictest0-21swapper/211:26:262
20556997652,19cyclictest23013-21kworker/u17:0+events_unbound10:20:482
20556997652,16cyclictest27765-21sh10:41:162
2054599766,38cyclictest15-21ksoftirqd/008:34:560
2054599766,38cyclictest15-21ksoftirqd/008:34:560
20545997652,16cyclictest18531-21apt-key08:25:520
20545997631,24cyclictest15-21ksoftirqd/007:39:350
20545997622,32cyclictest15-21ksoftirqd/009:48:000
20545997556,12cyclictest11839-21systemd-run08:05:500
20545997555,13cyclictest25360-21latency_hist08:45:510
20545997545,15cyclictest181rcu_preempt07:18:240
20545997536,16cyclictest0-21swapper/011:14:220
20545997531,18cyclictest181rcu_preempt09:28:010
2054599752,42cyclictest15-21ksoftirqd/008:00:410
20545997516,32cyclictest17541-21/usr/sbin/munin09:31:270
20556997456,11cyclictest31165-21switchtime10:46:242
20545997453,14cyclictest20899-21idleruntime-cro10:30:480
20545997446,10cyclictest181rcu_preempt09:00:430
2054599743,38cyclictest7929-21tail07:51:270
2054599742,22cyclictest15-21ksoftirqd/010:06:220
20556997353,14cyclictest18435-21sh09:31:252
2055699733,30cyclictest1366-21idleruntime-cro07:35:492
20556997329,11cyclictest181rcu_preempt07:36:252
20545997343,12cyclictest0-21swapper/010:36:290
2054599732,35cyclictest15-21ksoftirqd/010:04:210
2054599732,35cyclictest15-21ksoftirqd/010:04:210
20556997257,10cyclictest31061-21fschecks_count07:26:142
20556997253,13cyclictest18994-21sh10:26:252
20556997147,10cyclictest20532-21cyclictest08:16:142
20556997145,9cyclictest7215-21unixbench_multi10:06:282
2054599717,40cyclictest15-21ksoftirqd/011:03:210
20545997139,12cyclictest181rcu_preempt11:46:060
2054599713,40cyclictest23009-21awk12:21:160
20556997052,11cyclictest28115-21ssh09:47:332
20556997022,10cyclictest181rcu_preempt12:18:352
20545997051,13cyclictest29566-21ssh09:50:550
20545997039,10cyclictest181rcu_preempt07:21:170
2054599703,44cyclictest15-21ksoftirqd/010:48:310
20545997018,31cyclictest15-21ksoftirqd/007:11:050
20545997018,31cyclictest15-21ksoftirqd/007:11:050
20556996940,20cyclictest22057-21modprobe07:00:492
2054599696,28cyclictest20050-21rm12:16:180
2054599692,37cyclictest15-21ksoftirqd/008:51:090
2054599692,37cyclictest15-21ksoftirqd/008:51:090
20556996853,10cyclictest7999-21apt-get09:15:482
20556996851,11cyclictest2905-21ntp_kernel_pll_09:06:172
20556996851,11cyclictest2905-21ntp_kernel_pll_09:06:172
2055699684,46cyclictest22959-21fschecks_count07:01:132
2055699684,46cyclictest22959-21fschecks_count07:01:132
20556996840,19cyclictest25208-21switchtime10:36:272
20556996836,8cyclictest181rcu_preempt12:05:512
2055699683,44cyclictest13023-21fschecks_count11:11:102
20556996834,11cyclictest181rcu_preempt07:56:052
2055699683,33cyclictest3826-21/usr/sbin/munin10:56:302
20556996816,34cyclictest10450-21apt-get08:01:022
20545996810,30cyclictest15-21ksoftirqd/007:31:490
20556996742,10cyclictest0-21swapper/209:01:582
20563996619,40cyclictest0-21swapper/310:31:403
20563996619,40cyclictest0-21swapper/310:31:403
20556996651,10cyclictest7310-21sh11:01:162
20556996646,14cyclictest8343-21idleruntime-cro07:55:492
20556996636,8cyclictest181rcu_preempt09:55:502
20556996633,10cyclictest20095-21modprobe08:30:482
20556996542,17cyclictest25568-21cstates08:46:082
20556996536,20cyclictest32566-21unixbench_singl11:43:142
20556996536,20cyclictest32566-21unixbench_singl11:43:142
20556996441,9cyclictest58-21kcompactd009:00:402
20556996436,20cyclictest18232-21latency_hist08:25:532
20556996428,13cyclictest181rcu_preempt08:36:582
20556996428,13cyclictest181rcu_preempt08:36:582
2055699634,39cyclictest352-21runrttasks08:55:042
2055699634,39cyclictest352-21runrttasks08:55:042
2055699633,38cyclictest23884-21fschecks_time09:41:142
20556996239,14cyclictest0-21swapper/212:25:492
2055699623,25cyclictest15939-21sh11:16:102
20556996147,7cyclictest0-21swapper/212:11:262
20563996046,9cyclictest451rcuc/308:01:143
20556996027,13cyclictest58-21kcompactd008:11:112
2056399594,42cyclictest7274-21taskset07:51:133
20563995923,27cyclictest0-21swapper/311:41:103
20563995923,27cyclictest0-21swapper/311:41:103
2056399583,42cyclictest12462-21taskset12:03:463
2056399574,41cyclictest21091-21taskset08:31:233
2056399574,41cyclictest21091-21taskset08:31:233
20563995736,14cyclictest451rcuc/308:56:193
2056399568,30cyclictest0-21swapper/308:26:043
2056399563,36cyclictest4386-21taskset11:50:523
2056399559,32cyclictest0-21swapper/309:16:163
2056399553,38cyclictest4655-21chrt11:51:053
2056399553,38cyclictest4655-21chrt11:51:053
2056399553,38cyclictest13618-21taskset12:06:053
2056399553,36cyclictest8005-21chrt09:15:483
2056399553,34cyclictest24328-21chrt10:35:563
2056399553,34cyclictest1616-21taskset07:36:013
2056399544,34cyclictest7227-21chrt11:01:143
2056399543,37cyclictest19607-21chrt12:16:063
20563995410,31cyclictest0-21swapper/307:41:093
2056399534,36cyclictest12651-21taskset09:21:283
2056399533,37cyclictest7838-21chrt10:07:583
2056399533,37cyclictest23040-21taskset12:21:173
2056399532,40cyclictest23745-21taskset11:29:033
2056399532,40cyclictest23745-21taskset11:29:033
2056399532,38cyclictest29937-21chrt11:39:453
2056399532,38cyclictest29937-21chrt11:39:453
2056399532,38cyclictest18221-21chrt09:31:213
2056399532,37cyclictest16857-21chrt10:23:023
20563995313,28cyclictest14495-21chrt10:19:503
2056399525,32cyclictest31905-21chrt10:47:443
2056399523,35cyclictest16666-21chrt12:11:083
2056399523,33cyclictest23042-21chrt07:01:143
2056399523,33cyclictest23042-21chrt07:01:143
2056399522,33cyclictest28761-21chrt09:49:213
2056399522,27cyclictest0-21swapper/308:36:023
2056399522,27cyclictest0-21swapper/308:36:023
2056399515,34cyclictest19248-21taskset10:26:273
2056399514,36cyclictest7841-21chrt11:56:123
2056399514,32cyclictest28365-21chrt07:16:253
2056399514,31cyclictest29597-21chrt09:50:593
20563995137,8cyclictest451rcuc/308:54:523
20563995137,8cyclictest451rcuc/308:54:523
2056399513,35cyclictest14990-21taskset09:26:143
2056399513,35cyclictest13622-21chrt08:10:523
2056399513,34cyclictest25761-21chrt08:46:133
2056399513,34cyclictest10172-21chrt11:06:123
2056399513,33cyclictest2284-21chrt09:59:193
2056399513,30cyclictest18620-21chrt11:20:523
2056399512,34cyclictest2149-21chrt10:51:573
2056399512,34cyclictest2149-21chrt10:51:573
2056399503,35cyclictest20850-21chrt11:24:213
2056399503,33cyclictest22283-21chrt09:38:063
2056399502,39cyclictest3191-21chrt10:01:053
2056399502,39cyclictest3191-21chrt10:01:053
2056399493,34cyclictest27576-21chrt07:15:523
2056399493,29cyclictest11884-21chrt10:15:493
2056399493,29cyclictest11884-21chrt10:15:493
2056399492,35cyclictest9193-21chrt07:56:233
2056399492,35cyclictest24908-21chrt07:06:163
2056399492,35cyclictest24908-21chrt07:06:163
2056399492,34cyclictest1910-21taskset09:05:393
2056399484,35cyclictest26607-21chrt11:33:453
2056399483,36cyclictest23621-21taskset09:41:063
2056399473,31cyclictest715-21chrt07:31:233
20563994732,9cyclictest0-21swapper/307:46:153
2056399472,32cyclictest25141-21chrt08:45:373
288092465,10sleep30-21swapper/310:42:303
2056399464,32cyclictest17465-21chrt08:21:213
2056399464,31cyclictest5874-21chrt10:59:073
2056399463,34cyclictest14971-21taskset11:14:363
1907924614,19sleep30-21swapper/306:51:073
2056399453,33cyclictest29293-21taskset07:21:073
2056399443,29cyclictest30828-21chrt07:25:533
2056399442,30cyclictest26128-21chrt07:11:063
20563994125,9cyclictest451rcuc/308:08:173
2056399393,27cyclictest16490-21taskset08:19:183
2056399392,27cyclictest14902-21taskset08:14:323
2056399383,25cyclictest21267-21chrt06:56:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional