You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-01 - 07:49
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Sat Aug 01, 2026 00:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20363217810,26sleep00-21swapper/018:55:520
1587421220,7sleep3451rcuc/322:22:193
20524991113,83cyclictest3978-21sh23:51:072
1922021119,70sleep119269-21fschecks_time18:51:201
20524991063,67cyclictest15246-21apt-get00:11:052
2197021040,5sleep20-21swapper/221:40:192
205199910475,21cyclictest586-21wc23:45:541
20524991028,76cyclictest6411-21ssh23:01:332
205249910274,19cyclictest5671-21munin-plugin-st23:00:532
20524991003,35cyclictest24171-21latency_hist20:45:542
2052499993,40cyclictest11579-21tune2fs22:16:132
20524999767,20cyclictest25163-21sendmail21:45:512
2052499973,35cyclictest24640-21cut20:46:122
195922979,61sleep219607-21open_inodes18:51:342
20524999568,18cyclictest28499-21apt-key21:50:552
20524999542,45cyclictest0-21swapper/220:16:282
2052499944,35cyclictest21787-21ssh23:26:182
20524999437,14cyclictest9325-21ssh21:19:092
20524999437,14cyclictest9325-21ssh21:19:092
20524999435,51cyclictest0-21swapper/222:11:072
20524999434,8cyclictest0-21swapper/222:30:592
2052499943,37cyclictest15575-21ls23:16:142
2052499942,31cyclictest20947-21grep00:20:522
2052499933,35cyclictest2005-21cut19:36:182
2052499933,35cyclictest18066-21cpuspeed20:26:072
2052499933,31cyclictest10069-21head00:01:282
20524999326,18cyclictest0-21swapper/219:15:542
2052499932,35cyclictest22209-21grep00:21:262
2052499932,35cyclictest16942-21/usr/sbin/munin21:31:162
2052499932,34cyclictest21081-21latency_hist20:35:532
2052499932,34cyclictest21081-21latency_hist20:35:532
20524999241,9cyclictest0-21swapper/222:51:172
2052499922,36cyclictest13735-21uname20:11:242
2052499922,33cyclictest13895-21if_eth021:26:162
2052499914,34cyclictest5038-21awk21:11:252
20524999135,9cyclictest0-21swapper/223:11:222
2052499913,43cyclictest458-21sh21:56:432
2052499913,38cyclictest16540-21fschecks_count20:21:132
2052499913,37cyclictest352-21runrttasks20:06:322
2052499913,34cyclictest3955-21modprobe21:10:512
20524999124,14cyclictest0-21swapper/200:06:192
20524999124,14cyclictest0-21swapper/200:06:192
2052499912,35cyclictest28478-21ssh20:56:262
2052499904,34cyclictest30194-21awk19:21:262
2052499903,34cyclictest6279-21grep22:06:272
2052499903,34cyclictest6279-21grep22:06:272
2052499903,34cyclictest31979-21grep19:26:272
2052499902,29cyclictest26428-21kernelversion20:51:182
20515999068,16cyclictest2538-21fschecks_time22:01:140
2052499897,72cyclictest13914-21apt-get22:21:072
2052499893,39cyclictest22675-21latency_hist20:40:532
2052499893,38cyclictest23506-21timerandwakeup19:01:292
2052499893,35cyclictest17482-21/usr/sbin/munin22:26:002
2052499893,35cyclictest17482-21/usr/sbin/munin22:26:002
2052499893,34cyclictest23459-21acpi22:36:012
2052499892,36cyclictest29228-21ssh22:45:532
2052999883,37cyclictest0-21swapper/323:51:083
2052499889,38cyclictest10492-21apt-get21:21:072
2052499889,38cyclictest10492-21apt-get21:21:072
2052499883,41cyclictest7031-21runrttasks19:51:162
2052499883,40cyclictest591-21latency_hist23:45:542
2052499883,35cyclictest29994-21latency_hist23:40:522
2052499883,34cyclictest352-21runrttasks21:02:132
2052499883,34cyclictest28708-21/usr/sbin/munin21:51:472
2052499873,35cyclictest9992-21mailstats23:06:242
2052499872,36cyclictest6624-21latency_hist19:50:542
20519998752,10cyclictest170-21jbd2/mmcblk2p2-800:21:051
2052499863,33cyclictest3066-21wc22:01:272
2052499862,35cyclictest24697-21ssh23:31:172
2052499862,35cyclictest24697-21ssh23:31:172
2052499854,29cyclictest1342-21ntp_kernel_pll_23:46:202
2052499853,37cyclictest352-21runrttasks19:42:552
2052499853,37cyclictest352-21runrttasks19:42:552
20519998548,10cyclictest5071-21apt-get19:46:061
2052499842,39cyclictest8704-21unin-run00:00:512
20519998442,18cyclictest58-21kcompactd022:55:441
20515998450,27cyclictest31456-21ssh23:41:540
20515998424,36cyclictest0-21swapper/019:41:250
20515998424,36cyclictest0-21swapper/019:41:250
2052499832,35cyclictest352-21runrttasks20:03:472
2052499832,35cyclictest352-21runrttasks20:03:472
20519998366,12cyclictest0-21swapper/120:40:531
20519998344,14cyclictest20501-21cyclictest00:13:541
20515998366,11cyclictest27424-21diskmemload23:08:510
20515998339,35cyclictest0-21swapper/000:21:120
2052499822,35cyclictest29748-21fschecks_count22:46:132
2052499822,35cyclictest29748-21fschecks_count22:46:132
20515998247,7cyclictest181rcu_preempt21:49:480
2052499815,32cyclictest18189-21/usr/sbin/munin23:21:092
20519998151,8cyclictest181rcu_preempt22:03:031
20519998137,9cyclictest181rcu_preempt21:34:161
20519998044,12cyclictest58-21kcompactd022:22:471
20519998040,9cyclictest181rcu_preempt21:12:051
20515998058,15cyclictest31452-21latency19:26:160
152832800,9sleep00-21swapper/020:16:200
20519997957,15cyclictest12907-21processes23:11:231
20519997946,8cyclictest2317-21ssh23:47:491
20519997945,25cyclictest1552-21modprobe19:35:511
20515997959,7cyclictest181rcu_preempt00:10:540
20515997959,7cyclictest181rcu_preempt00:10:540
20515997958,14cyclictest7019-21sh23:56:240
2052499784,33cyclictest308-21cstates19:31:092
20519997842,9cyclictest0-21swapper/119:17:191
20519997825,12cyclictest181rcu_preempt23:56:051
20515997860,12cyclictest19477-21/usr/sbin/munin21:36:130
20515997856,17cyclictest20501-21cyclictest21:01:150
20515997856,16cyclictest18461-21sh00:16:130
20515997853,9cyclictest26408-21ssh22:40:550
20515997852,10cyclictest181rcu_preempt22:07:220
20515997852,10cyclictest181rcu_preempt22:07:220
20515997839,9cyclictest0-21swapper/019:15:530
2052499773,34cyclictest28749-21timerwakeupswit19:17:382
20519997724,12cyclictest181rcu_preempt00:06:281
20519997724,12cyclictest181rcu_preempt00:06:281
20515997758,7cyclictest181rcu_preempt19:05:550
20515997747,22cyclictest26991-21latency_hist23:35:530
20515997747,22cyclictest26991-21latency_hist23:35:530
20515997743,27cyclictest20501-21cyclictest22:56:240
20519997661,10cyclictest20501-21cyclictest22:48:401
20519997661,10cyclictest20501-21cyclictest22:48:401
20519997646,23cyclictest3282-21grep22:56:141
20519997636,10cyclictest181rcu_preempt20:55:551
20519997628,13cyclictest181rcu_preempt22:31:191
20515997658,8cyclictest0-21swapper/020:26:260
20515997649,17cyclictest12214-21/usr/sbin/munin23:11:190
20515997647,20cyclictest20217-21latency_hist22:30:530
20515997647,20cyclictest20217-21latency_hist22:30:530
2051599763,47cyclictest6223-21apt-get23:01:080
20519997561,10cyclictest58-21kcompactd020:27:491
20519997559,9cyclictest24069-21cron19:05:511
20519997542,10cyclictest181rcu_preempt21:40:181
20519997526,12cyclictest181rcu_preempt21:56:271
20519997512,12cyclictest301ktimers/123:06:031
20515997557,9cyclictest0-21swapper/020:50:540
20515997554,7cyclictest181rcu_preempt21:57:000
20515997552,8cyclictest181rcu_preempt21:51:350
2051599752,44cyclictest14464-21apt-get22:21:170
20519997439,9cyclictest181rcu_preempt23:26:121
20519997436,7cyclictest181rcu_preempt20:57:471
20519997427,12cyclictest181rcu_preempt22:16:031
20519997423,12cyclictest181rcu_preempt19:56:341
20515997459,8cyclictest0-21swapper/020:25:520
20515997457,8cyclictest0-21swapper/019:10:540
20515997455,9cyclictest0-21swapper/020:01:120
20515997455,9cyclictest0-21swapper/020:01:120
20515997452,9cyclictest181rcu_preempt21:32:360
20515997442,9cyclictest181rcu_preempt21:15:160
20515997436,19cyclictest58-21kcompactd000:05:380
20519997356,11cyclictest10867-21ssh00:03:371
20519997352,15cyclictest22958-21sh21:41:171
20519997341,12cyclictest181rcu_preempt20:16:511
20519997340,9cyclictest181rcu_preempt18:56:191
20519997339,9cyclictest181rcu_preempt20:15:511
20519997339,8cyclictest181rcu_preempt22:36:241
20519997338,9cyclictest181rcu_preempt19:21:271
20515997351,14cyclictest277-21dbus-daemon20:55:520
20515997349,8cyclictest181rcu_preempt21:25:440
20515997349,8cyclictest181rcu_preempt21:25:440
20515997346,14cyclictest15-21ksoftirqd/022:37:460
20524997253,13cyclictest25134-21threads19:06:272
2051999725,24cyclictest28708-21/usr/sbin/munin21:51:401
20519997249,17cyclictest12292-21threads20:06:221
20519997243,8cyclictest181rcu_preempt19:51:201
20519997242,9cyclictest181rcu_preempt20:21:021
20519997233,8cyclictest181rcu_preempt22:26:011
20519997233,8cyclictest181rcu_preempt22:26:011
20519997230,8cyclictest181rcu_preempt19:11:031
20515997257,9cyclictest20662-21diskmemload22:31:070
20515997253,13cyclictest0-21swapper/023:50:550
20515997250,8cyclictest15-21ksoftirqd/023:53:140
20515997249,8cyclictest181rcu_preempt22:46:170
20515997249,8cyclictest181rcu_preempt22:46:170
20519997157,9cyclictest16438-21kworker/u17:3-flush-0:3222:42:161
20519997139,11cyclictest181rcu_preempt21:29:511
20519997134,13cyclictest181rcu_preempt21:02:171
20519997131,16cyclictest181rcu_preempt23:02:101
20515997154,9cyclictest0-21swapper/022:11:210
20515997151,8cyclictest181rcu_preempt22:20:120
20515997147,8cyclictest181rcu_preempt19:34:510
20515997144,7cyclictest181rcu_preempt19:46:140
20519997057,9cyclictest58-21kcompactd020:04:401
20519997057,9cyclictest58-21kcompactd020:04:401
20519997050,13cyclictest25018-21switchtime23:31:261
20519997050,13cyclictest25018-21switchtime23:31:261
20519997035,8cyclictest181rcu_preempt23:36:251
20515997054,8cyclictest0-21swapper/020:10:530
20515997054,7cyclictest0-21swapper/019:00:540
20515997048,16cyclictest18444-21fschecks_count23:21:130
20515997047,9cyclictest15-21ksoftirqd/022:55:220
20515997040,11cyclictest181rcu_preempt20:56:270
128392700,9sleep30-21swapper/323:11:193
20519996924,8cyclictest181rcu_preempt21:46:201
20515996954,11cyclictest20501-21cyclictest23:36:140
20515996952,8cyclictest0-21swapper/019:36:180
20515996952,12cyclictest27424-21diskmemload23:16:080
20515996948,8cyclictest181rcu_preempt21:26:140
20515996947,14cyclictest0-21swapper/021:20:550
20515996947,14cyclictest0-21swapper/021:20:550
2052999684,51cyclictest28781-21taskset21:51:103
2052999682,34cyclictest0-21swapper/321:21:163
2052999682,34cyclictest0-21swapper/321:21:163
20519996851,11cyclictest5045-21ssh23:54:061
20519996838,21cyclictest23558-21grep20:41:261
20519996834,12cyclictest181rcu_preempt22:06:081
20519996834,12cyclictest181rcu_preempt22:06:081
20519996831,9cyclictest181rcu_preempt23:22:351
20515996853,8cyclictest0-21swapper/019:56:250
20515996853,8cyclictest0-21swapper/019:21:260
20515996849,8cyclictest0-21swapper/020:31:090
20515996849,8cyclictest0-21swapper/020:31:090
20515996841,18cyclictest16631-21unixbench_singl00:12:250
20519996739,8cyclictest10275-21ssh21:20:551
20519996739,8cyclictest10275-21ssh21:20:551
20519996732,8cyclictest181rcu_preempt21:10:541
20519996732,13cyclictest181rcu_preempt21:21:591
20519996732,13cyclictest181rcu_preempt21:21:591
20519996731,11cyclictest181rcu_preempt19:10:511
20515996752,8cyclictest0-21swapper/020:11:180
20515996752,10cyclictest22889-21ssh21:41:160
20515996726,11cyclictest1289-21apt-get21:06:060
2052999664,29cyclictest0-21swapper/323:01:323
2051999663,48cyclictest21182-21latency_hist20:35:541
20519996634,11cyclictest16319-21kworker/u19:2-rpciod23:18:501
20519996633,13cyclictest181rcu_preempt20:46:141
20519996629,8cyclictest181rcu_preempt00:20:531
20515996652,8cyclictest0-21swapper/019:51:170
20515996651,9cyclictest58-21kcompactd020:43:070
20515996642,15cyclictest29369-21latency_hist19:20:540
20515996639,21cyclictest4994-21kworker/u17:0+nfsiod23:26:570
20515996625,9cyclictest181rcu_preempt20:40:500
20524996513,17cyclictest21223-21ntp_offset18:56:202
20519996532,11cyclictest181rcu_preempt19:41:101
20519996532,11cyclictest181rcu_preempt19:41:101
2052999642,32cyclictest0-21swapper/319:16:233
20524996441,13cyclictest0-21swapper/219:56:192
20519996435,10cyclictest58-21kcompactd019:26:141
20519996430,13cyclictest181rcu_preempt20:35:541
20519996340,15cyclictest8467-21entropy22:11:111
20519996339,9cyclictest181rcu_preempt19:40:521
2052999592,39cyclictest917-21kworker/3:1+events20:31:233
2052999592,39cyclictest917-21kworker/3:1+events20:31:233
2052999573,37cyclictest15409-21chrt23:16:103
2052999572,40cyclictest10161-21chrt21:20:543
2052999572,40cyclictest10161-21chrt21:20:543
2052999563,40cyclictest18000-21taskset00:15:523
2052999553,34cyclictest3146-21taskset22:01:283
2052999552,36cyclictest917-21kworker/3:1+events21:03:483
2052999543,33cyclictest27067-21chrt20:51:313
2052999543,32cyclictest25050-21taskset23:31:263
2052999543,32cyclictest25050-21taskset23:31:263
2052999542,36cyclictest20755-21chrt22:31:113
2052999533,33cyclictest19807-21chrt22:29:533
2052999533,33cyclictest19807-21chrt22:29:533
2052999532,38cyclictest22922-21taskset21:41:173
2052999532,35cyclictest5303-21taskset19:46:133
2052999532,35cyclictest2584-21taskset22:55:353
2052999532,34cyclictest27651-21taskset23:36:163
2052999523,32cyclictest31025-21taskset22:47:403
2052999523,32cyclictest31025-21taskset22:47:403
2052999522,34cyclictest11824-21chrt20:06:103
2052999522,33cyclictest4007-21taskset21:10:523
2052999522,33cyclictest12400-21chrt22:16:353
2052999513,34cyclictest29886-21chrt20:59:253
2052999512,33cyclictest23541-21chrt19:01:293
2052999503,29cyclictest7231-21chrt23:56:513
2052999502,35cyclictest25990-21chrt19:11:053
2052999502,35cyclictest14242-21chrt20:12:503
2052999502,33cyclictest19415-21chrt23:21:453
2052999502,32cyclictest21246-21chrt20:36:073
2052999494,33cyclictest12313-21chrt00:06:163
2052999494,33cyclictest12313-21chrt00:06:163
2052999494,32cyclictest19679-21taskset21:36:103
2052999493,34cyclictest21247-21chrt23:26:033
2052999493,33cyclictest21438-21chrt00:21:113
2052999493,29cyclictest30535-21taskset23:41:143
2052999493,29cyclictest25985-21chrt20:50:543
2052999492,34cyclictest10578-21chrt22:14:533
2052999483,32cyclictest9866-21chrt23:06:213
2052999483,32cyclictest21518-21chrt18:56:273
2052999483,30cyclictest917-21kworker/3:1+events19:34:313
2052999482,33cyclictest9498-21chrt00:01:173
2052999482,29cyclictest16457-21chrt20:21:093
2052999473,32cyclictest4539-21chrt22:57:413
2052999463,27cyclictest917-21kworker/3:1+events19:21:073
2052999462,33cyclictest3454-21chrt19:41:073
2052999462,33cyclictest3454-21chrt19:41:073
2052999462,33cyclictest2669-21taskset23:48:483
2052999462,29cyclictest4791-21chrt21:11:173
2052999462,27cyclictest3335-21chrt19:40:543
191372469,21sleep30-21swapper/318:51:143
2052999453,28cyclictest13663-21taskset21:26:093
2052999453,27cyclictest31990-21taskset21:56:133
2052999453,27cyclictest16615-21chrt21:31:083
2052999443,27cyclictest11358-21chrt20:05:543
2052999443,27cyclictest11358-21chrt20:05:543
2052999442,27cyclictest29299-21chrt22:45:543
2052999433,28cyclictest20547-21chrt00:19:533
2052999433,25cyclictest23758-21chrt22:36:133
2052999432,29cyclictest8098-21taskset19:55:533
2052999422,27cyclictest32316-21chrt19:29:483
2052999414,23cyclictest26699-21chrt21:46:443
2052999413,27cyclictest5279-21chrt22:06:013
2052999413,27cyclictest5279-21chrt22:06:013
2052999413,27cyclictest18014-21chrt20:26:073
2052999393,26cyclictest8910-21taskset19:56:303
2052999383,26cyclictest14760-21chrt20:16:023
2052999382,27cyclictest23589-21taskset20:41:263
20529993821,9cyclictest0-21swapper/319:06:203
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional