You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 17:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Fri Dec 26, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
989324544,0sleep10-21swapper/107:09:591
977623828,0sleep30-21swapper/307:08:373
981623726,0sleep00-21swapper/007:09:060
972723627,0sleep20-21swapper/207:08:042
10062991010,0cyclictest0-21swapper/209:11:262
100609993,0cyclictest32614-21awk12:16:240
100609983,0cyclictest9343-21awk12:21:250
100609983,0cyclictest18344-21awk12:26:270
100609983,0cyclictest16418-21awk10:01:250
100609983,0cyclictest15062-21sadc10:01:090
100609981,0cyclictest0-21swapper/007:31:170
100609973,0cyclictest3957-21cut12:36:230
100609972,0cyclictest8480-21awk09:21:230
100609972,0cyclictest6811-21sed11:26:230
100609972,0cyclictest3524-21snmpd11:33:500
100609972,0cyclictest3524-21snmpd09:08:230
100609972,0cyclictest3524-21snmpd07:14:250
100609972,0cyclictest27239-21awk11:01:260
100609972,0cyclictest25706-21awk07:21:240
100609972,0cyclictest2398-21ssh09:36:070
100609972,0cyclictest23703-21awk12:11:250
100609972,0cyclictest21621-21tr08:26:220
100609971,0cyclictest32665-21awk10:46:250
100609971,0cyclictest15108-21awk07:36:260
100609970,0cyclictest4627-21kthreadcore11:41:310
100609970,0cyclictest3749-21tail11:06:210
100609970,0cyclictest32006-21latency12:31:290
100609970,0cyclictest29876-21ssh09:15:260
100609970,0cyclictest29677-21kthreadcore08:06:270
100609970,0cyclictest28392-21ssh10:26:210
100609970,0cyclictest27583-21kthreadcore11:36:280
100609970,0cyclictest25084-21sh11:51:330
100609970,0cyclictest24581-21cpuspeed_turbos10:06:220
100609970,0cyclictest22220-21sh10:41:010
100609970,0cyclictest18931-21ssh09:42:240
100609970,0cyclictest18503-21ssh10:20:420
100609970,0cyclictest18503-21ssh10:20:420
100609970,0cyclictest16505-21cat11:46:330
100609970,0cyclictest10035-21kthreadcore12:01:270
339960,0migration/33515-21sh11:59:293
339960,0migration/332726-21sh09:31:423
339960,0migration/330891-21sh10:45:153
339960,0migration/327678-21sh09:46:573
339960,0migration/322567-21sh11:32:493
339960,0migration/322468-21sh10:02:243
339960,0migration/310195-21sh09:37:583
329960,0watchdog/39400-21users12:36:333
29805060,0irq/28-eth0-rx-7925-21seq10:11:573
29805060,0irq/28-eth0-rx-32709-21fschecks_count08:56:233
29805060,0irq/28-eth0-rx-21692-21kthreadcore12:26:293
100619960,0cyclictest0-21swapper/110:06:221
100619960,0cyclictest0-21swapper/108:41:331
100609962,0cyclictest3571-21lldpd07:16:200
100609961,0cyclictest10796-21tr07:56:220
100609961,0cyclictest10482chrt10:11:210
100609961,0cyclictest0-21swapper/008:31:190
100609960,0cyclictest8126-21ssh10:51:120
100609960,0cyclictest7943-21tail08:16:250
100609960,0cyclictest7602-21munin-node09:01:220
100609960,0cyclictest5050-21grep10:31:230
100609960,0cyclictest3728-21cut08:36:220
100609960,0cyclictest3469-21tr07:51:220
100609960,0cyclictest32375-21tr08:11:210
100609960,0cyclictest32246-21date08:56:090
100609960,0cyclictest32111-21cut07:26:210
100609960,0cyclictest31447-21grep09:16:240
100609960,0cyclictest29716-21ssh11:21:180
100609960,0cyclictest29243-21gltestperf07:46:220
100609960,0cyclictest27822-21sh11:56:120
100609960,0cyclictest2724-21sh09:36:160
100609960,0cyclictest25020-21grep08:51:090
100609960,0cyclictest22564-21ssh10:41:170
100609960,0cyclictest22048-21grep09:46:250
100609960,0cyclictest21229-21entropy11:16:240
100609960,0cyclictest21128-21munin-node07:41:210
100609960,0cyclictest18990-21sh10:21:180
100609960,0cyclictest18477-21sort08:01:230
100609960,0cyclictest17901-21fschecks_time10:56:220
100609960,0cyclictest16719-21aten_r1power_en09:26:200
100609960,0cyclictest14622-21diskstats08:21:230
100609960,0cyclictest13619-21aten2.4-expect12:06:200
100609960,0cyclictest11734-21rm11:11:100
100609960,0cyclictest11098-21diskstats08:41:240
100609960,0cyclictest0-21swapper/009:51:210
100609960,0cyclictest0-21swapper/008:46:230
8305050,0irq/26-0000:00:32140-21latency_hist08:56:092
8305050,0irq/26-0000:00:31695-21latency_hist07:26:083
8305050,0irq/26-0000:00:0-21swapper/209:11:092
339950,0migration/39170-21sh10:51:243
339950,0migration/331870-21taskset08:54:213
339950,0migration/330416-21kthreadcore08:31:253
339950,0migration/330344-21missed_timers10:06:283
339950,0migration/327862-21kthreadcore12:11:283
339950,0migration/326456-21kthreadcore09:11:253
339950,0migration/326202-21kthreadcore11:36:283
339950,0migration/321526-21sh12:09:283
339950,0migration/320889-21taskset08:23:203
339950,0migration/317833-21aten_r1power_vo08:01:213
339950,0migration/317313-21kthreadcore10:36:283
339950,0migration/315872-21kthreadcore07:36:283
339950,0migration/312357-21fschecks_count09:41:243
339950,0migration/311042-21fschecks_time07:11:223
339950,0migration/310924-21fschecks_time10:16:243
339950,0migration/310924-21fschecks_time10:16:243
339950,0migration/310201-21taskset07:55:293
329950,0watchdog/38212-21aten2.4-expect12:21:213
329950,0watchdog/329812-21cut12:31:293
329950,0watchdog/327022-21missed_timers07:41:283
329950,0watchdog/316683-21python07:56:323
329950,0watchdog/311008-21python10:31:323
29815050,0irq/29-eth0-tx-8992-21kthreadcore09:21:252
29815050,0irq/29-eth0-tx-26357-21cut09:46:272
29815050,0irq/29-eth0-tx-2249-21kthreadcore09:51:282
29815050,0irq/29-eth0-tx-22022-21sh11:16:282
29815050,0irq/29-eth0-tx-17972-21sh09:41:312
29815050,0irq/29-eth0-tx-17178-21munin-node10:56:212
29815050,0irq/29-eth0-tx-12836-21timerandwakeup09:56:342
29815050,0irq/29-eth0-tx-10029-21munin-node10:16:262
29815050,0irq/29-eth0-tx-10029-21munin-node10:16:262
29815050,0irq/29-eth0-tx-0-21swapper/212:26:342
29815050,0irq/29-eth0-tx-0-21swapper/211:11:242
29815050,0irq/29-eth0-tx-0-21swapper/211:02:132
29815050,0irq/29-eth0-tx-0-21swapper/210:55:422
29805050,0irq/28-eth0-rx-6314-21runrttasks10:26:103
29805050,0irq/28-eth0-rx-3220-21df_inode11:06:203
29805050,0irq/28-eth0-rx-30185-21diskmemload11:21:233
29805050,0irq/28-eth0-rx-22620-21kthreadcore11:16:293
29805050,0irq/28-eth0-rx-13614-21runrttasks09:57:473
29805050,0irq/28-eth0-rx-0-21swapper/312:20:353
29805050,0irq/28-eth0-rx-0-21swapper/312:01:203
29805050,0irq/28-eth0-rx-0-21swapper/311:47:093
29805050,0irq/28-eth0-rx-0-21swapper/311:26:093
29805050,0irq/28-eth0-rx-0-21swapper/311:11:133
29805050,0irq/28-eth0-rx-0-21swapper/310:56:143
29805050,0irq/28-eth0-rx-0-21swapper/310:46:343
29805050,0irq/28-eth0-rx-0-21swapper/310:25:163
29805050,0irq/28-eth0-rx-0-21swapper/309:53:143
29805050,0irq/28-eth0-rx-0-21swapper/309:30:223
29805050,0irq/28-eth0-rx-0-21swapper/309:16:233
259950,0migration/28791-21mdstat11:06:262
259950,0migration/28584-21sh10:13:302
259950,0migration/26261-21sh12:17:222
259950,0migration/232400-21munin-node12:31:322
259950,0migration/231327-21sh10:07:102
259950,0migration/220432-21switchtime08:21:342
249950,0watchdog/27744-21cut07:51:282
249950,0watchdog/220592-21kthreadcore07:16:272
179950,0migration/19102-21switchtime12:36:321
179950,0migration/116466-21ntp_offset07:56:291
169950,0watchdog/115931-21sh09:25:121
169950,0watchdog/110905-21perl10:16:241
169950,0watchdog/110905-21perl10:16:241
100629955,0cyclictest0-21swapper/211:51:502
100629955,0cyclictest0-21swapper/210:36:382
100629955,0cyclictest0-21swapper/209:38:562
100629950,0cyclictest0-21swapper/212:14:142
100629950,0cyclictest0-21swapper/211:57:312
100629950,0cyclictest0-21swapper/211:36:332
100629950,0cyclictest0-21swapper/210:28:552
100619955,0cyclictest21384-21diskmemload09:31:221
339940,0migration/38209-21irqrtprio07:31:233
339940,0migration/35740-21kthreadcore08:36:273
339940,0migration/33553-21kthreadcore08:11:263
339940,0migration/322097-21kthreadcore08:46:253
339940,0migration/322017-21kthreadcore11:51:303
329940,0watchdog/315132-21perl09:06:243
329940,0watchdog/312304-21cut08:41:283
329940,0watchdog/30-21swapper/307:26:273
329940,0watchdog/30-21swapper/307:16:273
29815040,0irq/29-eth0-tx-0-21swapper/212:36:122
29815040,0irq/29-eth0-tx-0-21swapper/211:48:242
29815040,0irq/29-eth0-tx-0-21swapper/209:31:162
29815040,0irq/29-eth0-tx-0-21swapper/209:26:262
29815040,0irq/29-eth0-tx-0-21swapper/209:16:142
29805040,0irq/28-eth0-rx-0-21swapper/311:41:533
29805040,0irq/28-eth0-rx-0-21swapper/311:01:113
29805040,0irq/28-eth0-rx-0-21swapper/309:21:203
29805040,0irq/28-eth0-rx-0-21swapper/308:26:283
29805040,0irq/28-eth0-rx-0-21swapper/307:46:333
259940,0migration/27768-21kthreadcore08:36:282
259940,0migration/232007-21munin-node07:26:212
259940,0migration/23048-21kthreadcore11:41:302
259940,0migration/230345-21kthreadcore07:21:272
259940,0migration/22385-21kthreadcore08:56:272
259940,0migration/221129-21munin-node07:41:192
259940,0migration/220877-21kthreadcore10:21:282
259940,0migration/219125-21kthreadcore10:01:282
259940,0migration/217396-21runrttasks08:43:242
259940,0migration/216961-21kthreadcore12:06:272
259940,0migration/215559-21kthreadcore07:56:272
249940,0watchdog/27098-21perl11:26:242
249940,0watchdog/223399-21fschecks_time10:41:242
249940,0watchdog/20-21swapper/211:31:242
249940,0watchdog/20-21swapper/208:26:242
179940,0migration/1933-21kthreadcore10:26:261
179940,0migration/1694-21run-parts11:02:091
179940,0migration/15695-21kthreadcore11:06:241
179940,0migration/15346-21fschecks_time12:01:221
179940,0migration/131059-21rt-features08:06:301
179940,0migration/130199-21sh09:51:221
179940,0migration/129026-21fschecks_count08:31:221
179940,0migration/128185-21kthreadcore12:31:281
179940,0migration/127342-21kthreadcore07:21:261
179940,0migration/127051-21kthreadcore09:11:251
179940,0migration/121352-21kthreadcore11:51:301
179940,0migration/121037-21cpu09:46:211
179940,0migration/120712-21switchtime09:06:351
179940,0migration/120058-21munin-node10:21:261
179940,0migration/118870-21idleruntime08:46:231
179940,0migration/117210-21sh12:26:221
179940,0migration/115435-21kthreadcore12:06:261
179940,0migration/115403-21sshd10:52:421
179940,0migration/11477-21fschecks_count11:41:261
179940,0migration/114193-21cpuspeed_turbos07:36:231
179940,0migration/113650-21sh09:57:531
179940,0migration/110918-21kthreadcore12:21:271
179940,0migration/110736-21fschecks_count07:11:221
169940,0watchdog/18684-21kthreadcore11:26:281
169940,0watchdog/17880-21ssh10:51:081
169940,0watchdog/14530-21ssh10:31:121
169940,0watchdog/122805-21kthreadcore08:01:281
169940,0watchdog/122360-21munin-node11:16:281
169940,0watchdog/121697-21ssh11:31:321
169940,0watchdog/117293-21ssh10:56:201
169940,0watchdog/113023-1kworker/1:0H11:13:281
169940,0watchdog/111186-21irqcore11:46:241
169940,0watchdog/10-21swapper/112:11:121
169940,0watchdog/10-21swapper/110:41:241
169940,0watchdog/10-21swapper/109:19:321
169940,0watchdog/10-21swapper/108:36:281
169940,0watchdog/10-21swapper/108:26:241
169940,0watchdog/10-21swapper/108:21:281
169940,0watchdog/10-21swapper/107:41:281
100629944,0cyclictest0-21swapper/212:01:462
100629943,0cyclictest9278-21awk12:21:252
100629943,0cyclictest8061-21awk08:16:262
100629943,0cyclictest3524-21snmpd10:31:562
100629943,0cyclictest32650-21awk10:46:252
100629943,0cyclictest30787-21awk11:21:252
100629943,0cyclictest29695-21awk08:31:242
100629940,0cyclictest6750-21perf08:16:092
100629940,0cyclictest0-21swapper/209:03:302
100619943,0cyclictest8250-21awk09:01:251
100619943,0cyclictest8076-21awk08:16:261
100619943,0cyclictest4420-21awk07:51:261
100619943,0cyclictest3524-21snmpd12:16:271
100619943,0cyclictest29500-21awk07:46:241
100619943,0cyclictest14689-21awk10:36:261
329930,0watchdog/30-21swapper/309:01:403
29805030,0irq/28-eth0-rx-0-21swapper/308:16:233
29805030,0irq/28-eth0-rx-0-21swapper/308:06:283
259930,0migration/214018-21sh07:36:222
249930,0watchdog/20-21swapper/208:49:082
249930,0watchdog/20-21swapper/207:31:232
179930,0migration/111843-21runrttasks09:41:141
169930,0watchdog/121384-21diskmemload11:36:241
169930,0watchdog/121384-21diskmemload09:37:121
169930,0watchdog/120594-21kthreadcore07:16:271
169930,0watchdog/10-21swapper/111:56:161
169930,0watchdog/10-21swapper/110:11:281
169930,0watchdog/10-21swapper/109:26:241
169930,0watchdog/10-21swapper/108:15:481
169930,0watchdog/10-21swapper/107:26:271
100629932,0cyclictest25996-21awk08:06:252
100629932,0cyclictest21799-21grep08:01:272
100629932,0cyclictest11406-21awk07:11:242
100619933,0cyclictest7795-21awk07:31:221
100619932,0cyclictest3524-21snmpd11:23:561
100619932,0cyclictest16542-21awk10:01:261
100619932,0cyclictest1066-21irqcore08:56:261
100619930,0cyclictest0-21swapper/108:51:101
100629920,0cyclictest0-21swapper/207:46:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional