You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 16:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Mon May 04, 2026 12:44:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2629521610,19sleep30-21swapper/306:57:403
26137212013,76sleep126224-21grep06:57:391
26228211611,72sleep226294-21/usr/sbin/munin06:57:402
26775991153,31cyclictest27565-21apt-get11:32:101
267689910880,19cyclictest2374-21systemd-journal07:01:550
267759910255,40cyclictest544-21cron07:08:511
26775999432,49cyclictest0-21swapper/108:01:541
26775999257,25cyclictest8659-21sh10:07:131
2677599904,39cyclictest22486-21grep08:26:541
26768999061,21cyclictest24157-21latency_hist08:31:550
2677599894,40cyclictest4836-21grep11:47:241
26775998943,36cyclictest0-21swapper/111:13:381
26775998839,42cyclictest8637-21timerandwakeup07:42:271
26775998839,42cyclictest8637-21timerandwakeup07:42:271
26775998836,42cyclictest20994-21cron11:21:511
26775998836,42cyclictest20994-21cron11:21:511
2677599883,32cyclictest352-21runrttasks07:39:101
2677599882,30cyclictest28179-21seq12:27:061
2677599882,28cyclictest6173-21latency_hist10:56:541
2677599874,25cyclictest12613-21/usr/sbin/munin10:12:321
2677599874,25cyclictest12613-21/usr/sbin/munin10:12:321
26775998735,40cyclictest0-21swapper/111:22:261
2677599873,28cyclictest17463-21/usr/sbin/munin10:22:191
26768998758,12cyclictest181rcu_preempt08:42:030
2677599863,31cyclictest32720-21/usr/sbin/munin10:47:281
2677599863,29cyclictest7155-21taskset10:57:281
2677599863,25cyclictest24545-21cut11:27:131
2677599862,28cyclictest23757-21fschecks_count10:32:131
26785998549,10cyclictest181rcu_preempt07:17:062
2677599854,27cyclictest16107-21/usr/sbin/munin12:07:081
2677599853,30cyclictest29187-21cron10:41:511
2677599852,31cyclictest16522-21cron09:26:501
26775998511,12cyclictest0-21swapper/108:02:181
26768998557,15cyclictest15-21ksoftirqd/009:17:060
26768998557,15cyclictest15-21ksoftirqd/009:17:060
26785998452,11cyclictest181rcu_preempt10:37:062
26785998416,33cyclictest6881-21cat11:51:532
2677599847,60cyclictest29201-21apt-get10:42:071
2677599844,38cyclictest624-21ntpq08:57:201
2677599844,38cyclictest624-21ntpq08:57:201
26775998439,37cyclictest0-21swapper/108:12:271
2677599843,32cyclictest17687-21switchtime09:27:271
2677599843,31cyclictest22860-21/usr/sbin/munin09:37:261
2677599843,22cyclictest12307-21apt-get11:07:081
2677599842,31cyclictest26483-21fschecks_time08:38:231
26768998454,21cyclictest3268-21grep07:27:130
26768998443,26cyclictest13021-21ssh10:13:400
26768998443,26cyclictest13021-21ssh10:13:400
2677599834,25cyclictest24245-21/usr/sbin/munin08:32:151
2677599833,28cyclictest9807-21cpuspeed07:47:261
26775998330,47cyclictest352-21runrttasks09:34:281
26775998330,47cyclictest352-21runrttasks09:34:281
26768998361,15cyclictest16893-21switchtime08:07:260
26768998340,29cyclictest9933-21ssh11:02:210
2677599824,31cyclictest6007-21awk10:02:191
2677599824,31cyclictest6007-21awk10:02:191
2677599824,31cyclictest29130-21cut09:47:131
2677599824,31cyclictest29130-21cut09:47:131
2677599823,33cyclictest352-21runrttasks10:30:011
2677599823,31cyclictest11706-21cat09:17:261
2677599823,31cyclictest11706-21cat09:17:261
2677599823,30cyclictest2274-21latency_hist09:56:541
26775998232,42cyclictest0-21swapper/109:02:181
26775998232,41cyclictest0-21swapper/107:02:131
2677599822,33cyclictest2167-21ssh11:42:291
2677599822,30cyclictest10976-21switchtime11:57:251
2677599822,29cyclictest22955-21switchtime12:17:261
2677599822,28cyclictest352-21runrttasks07:53:571
26768998243,24cyclictest8472-21rm09:12:220
26768998241,26cyclictest22727-21rm11:23:180
26768998241,26cyclictest14314-21ssh12:02:400
26785998167,10cyclictest26759-21cyclictest10:42:522
26785998155,17cyclictest1478-21diskmemload09:17:262
26785998155,17cyclictest1478-21diskmemload09:17:262
2677599814,28cyclictest1900-21ntp_kernel_pll_07:22:201
2677599814,28cyclictest1900-21ntp_kernel_pll_07:22:201
2677599813,33cyclictest5381-21ssh10:01:551
2677599813,31cyclictest31918-21grep08:52:271
2677599813,29cyclictest9794-21ls11:02:171
2677599813,29cyclictest4621-21/usr/sbin/munin07:32:261
2677599813,28cyclictest20048-21taskset08:17:241
2677599813,28cyclictest14729-21tune2fs10:17:131
26775998131,41cyclictest0-21swapper/109:42:031
26775998130,43cyclictest0-21swapper/112:02:271
2677599812,39cyclictest25174-21ssh12:22:061
2677599812,32cyclictest30750-21latency_hist08:51:531
2677599812,28cyclictest969-21latency_hist11:41:541
2677599812,28cyclictest7110-21/usr/sbin/munin11:52:171
2676899813,56cyclictest27565-21apt-get11:32:080
26785998040,12cyclictest181rcu_preempt07:37:062
26785998039,12cyclictest181rcu_preempt12:13:362
2677599804,29cyclictest17064-21unixbench_multi08:07:281
2677599803,35cyclictest21544-21ssh12:16:041
2677599803,31cyclictest23971-21latency_hist08:31:551
2677599803,30cyclictest352-21runrttasks07:21:531
2677599803,28cyclictest6173-21sshd09:08:131
26768998064,11cyclictest28677-21latency_hist09:46:540
26768998060,14cyclictest352-21runrttasks11:07:250
26768998053,8cyclictest181rcu_preempt12:20:320
26768998053,19cyclictest3886-21latency_hist11:46:530
26768998042,7cyclictest181rcu_preempt10:51:530
26785997963,10cyclictest18448-21ssh10:22:272
26785997944,14cyclictest181rcu_preempt11:57:012
26785997929,13cyclictest181rcu_preempt07:07:162
2677599793,33cyclictest7820-21/usr/sbin/munin09:12:021
2677599792,30cyclictest30926-21chrt07:12:141
26785997835,18cyclictest181rcu_preempt11:07:212
2677599782,32cyclictest2986-21/usr/sbin/munin07:27:151
2677599782,26cyclictest28339-21wc08:42:221
26768997864,9cyclictest30652-21ssh10:43:160
26768997858,14cyclictest7619-21latency_hist09:11:540
26785997757,14cyclictest30795-21fschecks_count11:37:122
26785997757,14cyclictest27436-21latency_hist11:31:552
26785997755,11cyclictest16771-21open_files12:07:242
26785997747,13cyclictest181rcu_preempt09:57:062
26768997762,9cyclictest12823-21sh12:01:500
26768997756,14cyclictest1293-21latency_hist07:21:550
26768997749,10cyclictest181rcu_preempt08:38:150
26785997662,10cyclictest170-21jbd2/mmcblk2p2-807:51:542
26785997660,10cyclictest22915-21sh09:37:072
26785997657,12cyclictest26344-21ssh10:36:542
26785997645,22cyclictest30239-21cron11:36:512
26768997662,10cyclictest26759-21cyclictest10:52:420
26768997661,10cyclictest26759-21cyclictest07:46:520
26768997661,10cyclictest26759-21cyclictest07:46:520
26768997658,9cyclictest0-21swapper/010:27:130
26768997658,9cyclictest0-21swapper/007:26:550
26768997658,9cyclictest0-21swapper/007:26:550
26768997655,8cyclictest15-21ksoftirqd/010:58:150
26768997654,8cyclictest181rcu_preempt08:59:140
26768997654,8cyclictest181rcu_preempt08:59:140
26768997653,16cyclictest15942-21latency_hist08:06:530
26785997556,14cyclictest2075-21fschecks_count09:02:142
26785997545,8cyclictest181rcu_preempt11:20:182
26785997545,8cyclictest181rcu_preempt11:20:182
26768997559,9cyclictest23756-21sh10:32:130
26768997555,16cyclictest26759-21cyclictest09:34:310
26768997555,16cyclictest26759-21cyclictest09:34:310
26768997550,9cyclictest181rcu_preempt08:22:160
26785997454,13cyclictest9432-21kworker/u18:2+events_unbound10:56:512
26785997452,16cyclictest1478-21diskmemload11:02:052
26785997451,17cyclictest25739-21latency_hist08:36:542
26768997460,9cyclictest26759-21cyclictest07:47:130
26768997457,12cyclictest18018-21fschecks_count08:12:140
26768997456,8cyclictest0-21swapper/008:17:260
26768997455,9cyclictest0-21swapper/010:22:190
26768997455,7cyclictest181rcu_preempt07:07:020
26768997451,9cyclictest181rcu_preempt08:51:280
26768997450,8cyclictest181rcu_preempt11:27:240
26785997356,10cyclictest0-21swapper/209:36:512
26785997356,10cyclictest0-21swapper/209:36:512
26785997355,12cyclictest277-21dbus-daemon12:31:522
26785997339,8cyclictest181rcu_preempt07:57:142
26785997337,11cyclictest181rcu_preempt10:30:302
26768997354,13cyclictest14579-21cpuspeed10:17:080
26768997353,8cyclictest181rcu_preempt11:38:000
26768997351,10cyclictest181rcu_preempt09:52:130
26768997349,18cyclictest18947-21ssh11:17:210
26768997349,18cyclictest18947-21ssh11:17:210
26768997349,16cyclictest21740-21ssh12:16:370
26768997346,8cyclictest181rcu_preempt10:11:550
26768997341,14cyclictest181rcu_preempt07:52:270
26785997241,9cyclictest181rcu_preempt08:42:042
26785997236,9cyclictest181rcu_preempt07:32:062
26785997236,12cyclictest181rcu_preempt12:22:082
26768997250,9cyclictest181rcu_preempt11:47:200
26785997154,9cyclictest0-21swapper/211:56:522
26785997153,12cyclictest14793-21ssh12:04:012
26785997137,9cyclictest181rcu_preempt09:27:102
26785997136,14cyclictest181rcu_preempt11:42:142
26768997153,8cyclictest0-21swapper/010:02:150
26768997153,8cyclictest0-21swapper/010:02:150
26768997151,14cyclictest25604-21latency_hist08:36:550
26768997149,7cyclictest181rcu_preempt07:06:540
26785997054,10cyclictest31698-21latency_hist09:51:542
26785997054,10cyclictest31698-21latency_hist09:51:542
26785997053,10cyclictest23211-21processes08:27:232
26785997041,8cyclictest0-21swapper/207:22:132
26785997041,8cyclictest0-21swapper/207:22:132
26785997041,20cyclictest15889-21modprobe08:06:512
26785997039,8cyclictest181rcu_preempt10:51:522
26785997032,8cyclictest181rcu_preempt09:52:032
26785997015,30cyclictest12235-21tail07:52:272
26768997055,9cyclictest29515-21diskmemload12:28:120
26768997053,11cyclictest15374-21ssh11:12:080
26768997051,10cyclictest0-21swapper/009:47:130
26768997051,10cyclictest0-21swapper/009:47:130
26768997051,10cyclictest0-21swapper/009:02:140
26768997046,8cyclictest181rcu_preempt08:52:120
26768997045,10cyclictest181rcu_preempt12:26:550
26768997035,13cyclictest181rcu_preempt07:37:260
26785996953,10cyclictest277-21dbus-daemon12:21:522
26785996945,18cyclictest8055-21fschecks_count07:42:132
26785996945,18cyclictest8055-21fschecks_count07:42:132
26768996955,8cyclictest0-21swapper/007:57:260
26768996954,9cyclictest14365-21ssh09:22:180
26768996946,10cyclictest181rcu_preempt07:32:290
26768996932,11cyclictest0-21swapper/009:30:170
26785996852,12cyclictest26759-21cyclictest07:12:202
26785996852,10cyclictest30679-21latency_hist08:51:532
26785996850,11cyclictest0-21swapper/209:46:512
26785996850,10cyclictest6802-21latency10:57:192
26768996846,7cyclictest181rcu_preempt09:40:510
26768996845,7cyclictest181rcu_preempt12:11:540
26785996751,10cyclictest20337-21unixbench_multi08:17:272
2678599674,48cyclictest11814-21fschecks_time10:12:152
2678599674,48cyclictest11814-21fschecks_time10:12:152
26785996743,8cyclictest181rcu_preempt08:52:202
26785996737,10cyclictest181rcu_preempt10:09:102
2678599673,34cyclictest21300-21/usr/sbin/munin11:22:082
26785996728,12cyclictest181rcu_preempt10:03:492
26785996728,12cyclictest181rcu_preempt10:03:492
2678599672,28cyclictest6512-21taskset09:09:082
26768996741,11cyclictest181rcu_preempt11:52:210
2679199664,45cyclictest29302-21taskset08:46:553
26785996652,10cyclictest58-21kcompactd008:22:122
26785996646,15cyclictest26759-21cyclictest09:25:102
2678599663,45cyclictest10692-21latency_hist09:16:542
2678599663,33cyclictest15444-21ssh10:17:282
26785996628,9cyclictest181rcu_preempt07:31:512
26768996646,14cyclictest29197-21latency_hist10:41:540
26768996638,12cyclictest181rcu_preempt07:12:140
26768996621,13cyclictest0-21swapper/009:57:270
26785996446,12cyclictest18933-21sshd08:14:132
26785996438,9cyclictest170-21jbd2/mmcblk2p2-808:38:012
26785996335,10cyclictest58-21kcompactd011:12:042
26785996334,8cyclictest181rcu_preempt08:57:052
26785996334,8cyclictest181rcu_preempt08:57:052
2678599633,39cyclictest27215-21fschecks_time07:02:142
26785996246,11cyclictest0-21swapper/208:07:262
243552610,7sleep30-21swapper/311:27:083
2679199585,38cyclictest3163-21chrt07:27:113
2679199583,34cyclictest26996-21chrt07:02:083
2679199573,42cyclictest1318-21chrt11:42:113
2679199564,36cyclictest22877-21chrt09:37:023
2679199545,39cyclictest22665-21chrt09:36:543
2679199545,39cyclictest22665-21chrt09:36:543
2679199544,39cyclictest0-21swapper/307:47:133
2679199544,37cyclictest11168-21chrt10:11:513
2679199543,40cyclictest5459-21chrt10:02:023
2679199543,40cyclictest5459-21chrt10:02:023
2679199543,38cyclictest19186-21chrt12:12:063
2679199543,37cyclictest18529-21chrt11:17:113
2679199543,37cyclictest18529-21chrt11:17:113
2679199543,37cyclictest12485-21chrt11:07:103
2679199543,36cyclictest17002-21chrt09:27:093
2679199542,35cyclictest985-21chrt09:53:393
2679199533,37cyclictest18703-21chrt12:11:333
2679199533,34cyclictest30737-21chrt10:43:203
2679199524,35cyclictest14949-21chrt08:02:183
2679199524,35cyclictest11477-21chrt11:58:083
2679199524,33cyclictest7256-21chrt09:11:133
2679199523,39cyclictest28473-21chrt12:27:133
2679199523,37cyclictest23798-21chrt12:18:593
2679199523,36cyclictest11081-21chrt09:17:123
2679199523,36cyclictest11081-21chrt09:17:123
2679199522,34cyclictest16956-21chrt11:13:553
2679199522,33cyclictest24643-21chrt10:32:343
2679199513,35cyclictest27154-21chrt10:37:243
2679199513,34cyclictest27743-21taskset08:42:053
2679199513,34cyclictest14536-21chrt12:03:193
2679199512,36cyclictest17915-21taskset08:12:093
2679199504,37cyclictest18265-21chrt10:22:253
2679199503,33cyclictest490-21chrt10:47:133
2679199503,33cyclictest13226-21chrt10:14:143
2679199503,33cyclictest13226-21chrt10:14:143
2679199498,32cyclictest10582-21chrt11:02:573
2679199494,34cyclictest31295-21taskset07:12:253
2679199493,33cyclictest5023-21chrt10:01:343
2679199493,33cyclictest21835-21chrt11:22:183
2679199493,33cyclictest21782-21chrt08:22:263
2679199493,31cyclictest16647-21chrt08:07:203
2679199493,30cyclictest16120-21taskset09:25:433
2679199492,37cyclictest29603-21chrt07:07:263
2679199492,33cyclictest2896-21chrt09:02:303
2679199492,31cyclictest29496-21chrt11:34:463
2679199484,34cyclictest8983-21taskset09:12:353
2679199483,28cyclictest2848-21chrt07:26:543
2679199483,28cyclictest2848-21chrt07:26:543
2679199482,32cyclictest4852-21taskset07:32:133
2679199482,31cyclictest8426-21taskset11:00:173
26791994810,28cyclictest29083-21chrt09:47:113
26791994810,28cyclictest29083-21chrt09:47:113
2679199473,34cyclictest5632-21chrt10:55:523
2679199473,30cyclictest6699-21chrt07:37:193
2679199463,33cyclictest22628-21chrt10:30:233
2679199463,32cyclictest28803-21taskset09:46:553
2679199463,31cyclictest9546-21taskset07:46:553
2679199463,31cyclictest9546-21taskset07:46:553
2679199462,35cyclictest6623-21chrt11:51:233
2679199462,35cyclictest32562-21chrt07:17:143
2679199462,34cyclictest22674-21chrt08:27:073
2679199454,29cyclictest32527-21chrt08:56:553
2679199453,32cyclictest27691-21taskset12:26:283
2679199453,30cyclictest25462-21taskset08:35:133
2679199452,31cyclictest11600-21chrt07:52:133
2679199444,31cyclictest27178-21chrt08:38:373
2679199443,32cyclictest31080-21chrt11:37:173
2679199443,31cyclictest16775-21chrt10:20:463
2679199434,26cyclictest7129-21taskset11:52:003
2679199416,26cyclictest13529-21taskset07:57:243
2679199413,28cyclictest32592-21taskset08:57:073
2679199413,28cyclictest32592-21taskset08:57:073
2679199403,27cyclictest19492-21chrt08:17:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional