You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-02 - 07:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Thu Apr 02, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3079924637,0sleep20-21swapper/219:18:162
3069624129,0sleep10-21swapper/119:17:031
3051623925,0sleep00-21swapper/019:14:580
3077523425,0sleep30-21swapper/319:17:573
309479999,0cyclictest3524-21snmpd23:55:230
339970,0migration/314687-21timerwakeupswit00:24:353
339960,0migration/37244-21kthreadcore23:44:283
339960,0migration/330660-21sh23:02:263
339960,0migration/310767-21sh22:31:093
329960,0watchdog/33038-21grep19:44:203
309489960,0cyclictest0-21swapper/123:54:221
29815060,0irq/29-eth0-tx-6208-21munin-node20:09:280
29815060,0irq/29-eth0-tx-6016-21munin-node19:24:210
29805060,0irq/28-eth0-rx-7425-21ntp_offset22:09:293
149960,0migration/06483-21sh21:51:460
149960,0migration/05029-21sh19:19:320
149960,0migration/03967-21kthreadcore20:04:280
149960,0migration/02707-21sh23:42:210
8305050,0irq/26-0000:00:31112-21wc19:19:092
339950,0migration/35379-21timerwakeupswit20:04:343
339950,0migration/33449-21pmu-power23:59:313
339950,0migration/331599-21munin-node20:49:203
339950,0migration/3307-21kthreadcore23:39:283
339950,0migration/330269-21fschecks_time22:44:233
339950,0migration/327572-21latency_hist19:39:093
339950,0migration/326918-21kthreadcore20:44:273
339950,0migration/326101-21munin-node23:19:233
339950,0migration/325941-21kthreadcore23:54:273
339950,0migration/325505-21munin-node21:24:363
339950,0migration/323176-21sendmail_mailtr00:29:313
339950,0migration/321077-21kthreadcore22:19:273
339950,0migration/319494-21switchtime19:29:323
339950,0migration/315022-21fschecks_time22:54:223
339950,0migration/311199-21gltestperf21:19:243
329950,0watchdog/35665-21grep23:24:293
329950,0watchdog/3216822sleep300:10:413
329950,0watchdog/321612-21cut23:14:293
329950,0watchdog/319654-21sensors20:14:323
329950,0watchdog/316889-40yum-updatesd-he22:15:203
309499955,0cyclictest0-21swapper/223:19:022
309499955,0cyclictest0-21swapper/222:04:182
309499955,0cyclictest0-21swapper/200:29:592
309499955,0cyclictest0-21swapper/200:02:422
309499954,0cyclictest3524-21snmpd22:18:062
309499950,0cyclictest0-21swapper/223:29:072
309499950,0cyclictest0-21swapper/221:55:472
309499950,0cyclictest0-21swapper/220:44:412
309499950,0cyclictest0-21swapper/219:34:132
309489955,0cyclictest0-21swapper/100:27:291
309489950,0cyclictest0-21swapper/123:09:451
309489950,0cyclictest0-21swapper/121:26:011
29815050,0irq/29-eth0-tx-8191-21munin-node00:24:212
29815050,0irq/29-eth0-tx-3510-21head22:29:222
29815050,0irq/29-eth0-tx-27867-21munin-node20:24:250
29815050,0irq/29-eth0-tx-26929-21mdstat23:54:282
29815050,0irq/29-eth0-tx-0-21swapper/222:59:272
29815050,0irq/29-eth0-tx-0-21swapper/222:57:572
29815050,0irq/29-eth0-tx-0-21swapper/020:34:240
29805050,0irq/28-eth0-rx-9284-21cut23:09:263
29805050,0irq/28-eth0-rx-28393-21df_abs21:29:223
29805050,0irq/28-eth0-rx-25295-21cut21:44:263
29805050,0irq/28-eth0-rx-24993-21cut22:39:273
29805050,0irq/28-eth0-rx-24635-21irqcore00:14:253
29805050,0irq/28-eth0-rx-17576-21kthreadcore21:39:283
29805050,0irq/28-eth0-rx-16409-21ssh21:59:213
29805050,0irq/28-eth0-rx-15841-21ssh21:58:503
29805050,0irq/28-eth0-rx-14395-21fschecks_time20:59:223
29805050,0irq/28-eth0-rx-0-21swapper/322:49:273
29805050,0irq/28-eth0-rx-0-21swapper/322:24:503
29805050,0irq/28-eth0-rx-0-21swapper/322:07:113
29805050,0irq/28-eth0-rx-0-21swapper/321:50:433
29805050,0irq/28-eth0-rx-0-21swapper/300:46:513
29805050,0irq/28-eth0-rx-0-21swapper/300:37:483
259950,0migration/228145-21aten_r1power_fr21:29:212
249950,0watchdog/22272-21if_err_eth1.322:09:242
19150,0ktimersoftd/10-21swapper/122:51:381
19150,0ktimersoftd/10-21swapper/122:09:021
19150,0ktimersoftd/10-21swapper/122:03:321
19150,0ktimersoftd/10-21swapper/119:59:361
19150,0ktimersoftd/10-21swapper/100:35:411
179950,0migration/19776-21latency_hist20:34:091
179950,0migration/18862-21kthreadcore00:04:281
179950,0migration/1401-21munin-node21:49:261
179950,0migration/128947-21fschecks_time21:29:231
179950,0migration/127096-21kthreadcore22:59:271
179950,0migration/12189-21munin-node00:39:241
179950,0migration/120342-21munin-node21:39:361
179950,0migration/117920-21idleruntime19:54:241
179950,0migration/111631-21kthreadcore19:24:261
179950,0migration/110445-21cpuspeed_turbos23:29:221
149950,0migration/09873-21kthreadcore21:54:270
149950,0migration/07756-21latency00:39:300
149950,0migration/03430-21kthreadcore23:24:280
149950,0migration/03383-21gltestperf19:44:200
149950,0migration/030970-21taskset20:45:100
149950,0migration/02988-21aten_r1power_en21:14:200
149950,0migration/028145-21kthreadcore21:44:270
149950,0migration/027361-21kthreadcore00:34:250
149950,0migration/027287-21aten_r1power_fr22:24:220
149950,0migration/026657-21latency21:04:280
149950,0migration/025965-21sh23:35:580
149950,0migration/025540-21kthreadcore00:14:260
149950,0migration/022243-21kthreadcore19:34:250
149950,0migration/021516-21kthreadcore00:29:270
149950,0migration/021222-21kthreadcore21:59:270
149950,0migration/018470-21kthreadcore20:59:260
149950,0migration/015633-21kthreadcore22:34:270
149950,0migration/014757-21kthreadcore23:29:270
149950,0migration/013922-21fschecks_time19:29:210
339940,0migration/34460-21irqrtprio21:14:263
339940,0migration/332285-21if_eth119:19:253
339940,0migration/328738-21fschecks_time21:09:233
329940,0watchdog/30-21swapper/323:04:253
329940,0watchdog/30-21swapper/322:34:243
329940,0watchdog/30-21swapper/321:04:283
329940,0watchdog/30-21swapper/320:54:203
329940,0watchdog/30-21swapper/319:34:243
329940,0watchdog/30-21swapper/300:39:293
309499944,0cyclictest11721-21irqcore00:44:262
309499943,0cyclictest3571-21lldpd22:44:252
309499943,0cyclictest3571-21lldpd22:19:252
309499943,0cyclictest3571-21lldpd21:19:252
309499943,0cyclictest3571-21lldpd20:49:252
309499943,0cyclictest3571-21lldpd20:29:552
309499943,0cyclictest3571-21lldpd20:24:252
309499943,0cyclictest3571-21lldpd20:12:252
309499943,0cyclictest3571-21lldpd20:03:542
309499943,0cyclictest3571-21lldpd19:49:242
309499943,0cyclictest3571-21lldpd19:41:242
309499943,0cyclictest3571-21lldpd19:30:232
309499943,0cyclictest32751-21awk23:04:262
309499943,0cyclictest3250-21irqbalance21:24:162
309499943,0cyclictest3250-21irqbalance21:15:262
309499943,0cyclictest3250-21irqbalance21:10:362
309499943,0cyclictest3250-21irqbalance21:04:262
309499943,0cyclictest3250-21irqbalance20:59:262
309499943,0cyclictest21796-21awk20:19:232
309499943,0cyclictest11278-21awk23:29:252
309489944,0cyclictest0-21swapper/123:43:581
309489943,0cyclictest9509-21sadc19:49:091
309489943,0cyclictest8768-21awk21:54:251
309489943,0cyclictest25526-21awk20:44:261
309489943,0cyclictest21484-21awk19:34:231
309489943,0cyclictest14280-21awk19:29:231
309489943,0cyclictest11111-21awk22:14:281
309479944,0cyclictest0-21swapper/000:45:500
309479943,0cyclictest7609-21awk20:54:240
309479943,0cyclictest6821-21awk22:49:250
309479943,0cyclictest4791-21awk23:44:250
309479943,0cyclictest4551-21awk22:29:260
309479943,0cyclictest3874-21awk20:29:250
309479943,0cyclictest3524-21snmpd22:11:360
309479943,0cyclictest3524-21snmpd20:39:200
309479943,0cyclictest3524-21snmpd20:23:350
309479943,0cyclictest32663-21awk20:49:240
309479943,0cyclictest28787-21awk19:39:240
309479943,0cyclictest25194-21awk19:59:260
309479943,0cyclictest24122-21awk22:59:250
309479943,0cyclictest10895-21awk19:49:240
29815040,0irq/29-eth0-tx-5654-21ssh22:49:112
29815040,0irq/29-eth0-tx-420-21ssh00:19:222
29815040,0irq/29-eth0-tx-32196-21ssh23:20:272
29815040,0irq/29-eth0-tx-24129-21perl00:14:232
29815040,0irq/29-eth0-tx-15132-21ssh23:10:572
29815040,0irq/29-eth0-tx-13507-21ssh21:39:102
29815040,0irq/29-eth0-tx-0-21swapper/223:44:132
29815040,0irq/29-eth0-tx-0-21swapper/222:40:152
29815040,0irq/29-eth0-tx-0-21swapper/222:35:092
29815040,0irq/29-eth0-tx-0-21swapper/222:24:402
29815040,0irq/29-eth0-tx-0-21swapper/221:44:212
29815040,0irq/29-eth0-tx-0-21swapper/221:34:182
29815040,0irq/29-eth0-tx-0-21swapper/200:39:202
29815040,0irq/29-eth0-tx-0-21swapper/200:09:162
29815040,0irq/29-eth0-tx-0-21swapper/200:04:432
29815040,0irq/29-eth0-tx-0-21swapper/020:14:210
29805040,0irq/28-eth0-rx-19973-21iostat_ios23:34:253
29805040,0irq/28-eth0-rx-0-21swapper/323:49:093
29805040,0irq/28-eth0-rx-0-21swapper/323:29:423
29805040,0irq/28-eth0-rx-0-21swapper/321:34:233
29805040,0irq/28-eth0-rx-0-21swapper/320:34:273
29805040,0irq/28-eth0-rx-0-21swapper/320:09:213
29805040,0irq/28-eth0-rx-0-21swapper/319:54:213
29805040,0irq/28-eth0-rx-0-21swapper/319:52:593
29805040,0irq/28-eth0-rx-0-21swapper/300:20:313
29805040,0irq/28-eth0-rx-0-21swapper/300:04:213
259940,0migration/227850-21munin-node23:39:232
259940,0migration/226687-21if_eth1.300:34:242
259940,0migration/220721-21kthreadcore20:39:262
259940,0migration/217680-21fschecks_time19:54:232
259940,0migration/214625-21kthreadcore23:49:262
259940,0migration/212447-21munin-node20:54:272
249940,0watchdog/231348-21cut21:49:202
249940,0watchdog/218440-21kthreadcore20:14:282
249940,0watchdog/20-21swapper/223:34:212
249940,0watchdog/20-21swapper/221:59:202
249940,0watchdog/20-21swapper/219:44:242
179940,0migration/18838-21kthreadcore20:54:251
179940,0migration/14905-21kthreadcore20:49:271
179940,0migration/131674-21timerandwakeup23:19:341
179940,0migration/130891-21kthreadcore20:24:261
179940,0migration/130060-21kthreadcore21:09:271
179940,0migration/127776-21aten_r1power_vo19:39:201
179940,0migration/127011-21kthreadcore00:14:271
179940,0migration/126460-21kthreadcore21:44:271
179940,0migration/124171-21kthreadcore22:19:281
179940,0migration/123290-21kthreadcore20:19:251
179940,0migration/122733-21kthreadcore00:29:271
179940,0migration/122244-21irqcore21:04:271
179940,0migration/121329-21kthreadcore23:34:261
179940,0migration/118656-21irqrtprio20:39:251
179940,0migration/116690-21switchtime21:19:351
179940,0migration/116044-21irqrtprio00:09:261
179940,0migration/115863-21kthreadcore20:59:261
179940,0migration/114708-21kthreadcore22:34:271
179940,0migration/113440-21runrttasks20:14:171
179940,0migration/113199-21taskset21:38:591
179940,0migration/112912-21runrttasks20:10:451
179940,0migration/111487-21kthreadcore19:49:251
169940,0watchdog/19688-21memory23:44:291
169940,0watchdog/132709-21perl00:19:211
169940,0watchdog/132590-21idleruntime23:04:251
169940,0watchdog/131520-21kthreadcore22:24:281
169940,0watchdog/113579-21iostat23:49:251
169940,0watchdog/110555-21aten_r1power_fr00:44:211
169940,0watchdog/10-21swapper/123:14:291
149940,0migration/06263-21irqstats21:34:260
149940,0migration/06023-21taskset23:04:440
149940,0migration/031523-21kthreadcore23:59:270
149940,0migration/022435-21runrttasks22:58:290
149940,0migration/017417-21sh22:16:470
339930,0migration/38593-21latency20:29:283
329930,0watchdog/3298050irq/28-eth0-rx-20:19:203
329930,0watchdog/30-21swapper/320:24:123
329930,0watchdog/30-21swapper/319:24:203
309499932,0cyclictest3571-21lldpd19:25:232
309499932,0cyclictest31588-21cpuspeed_turbos20:04:222
309499932,0cyclictest11040-21awk20:34:242
309489933,0cyclictest0-21swapper/121:14:211
309489933,0cyclictest0-21swapper/120:04:211
309489932,0cyclictest3524-21snmpd20:38:061
309489932,0cyclictest2377-21awk22:09:251
309489932,0cyclictest15416-21awk22:54:241
309489930,0cyclictest5006-21runrttasks22:47:591
309489930,0cyclictest3350-21latency_hist22:29:101
309489930,0cyclictest2632-21kthreadcore23:59:281
309479932,0cyclictest522-21awk00:19:230
309479932,0cyclictest31771-21ssh22:04:580
309479932,0cyclictest30558-21awk22:44:250
309479932,0cyclictest21114-21cut23:14:280
309479932,0cyclictest18961-21df_abs22:19:220
309479932,0cyclictest18000-21awk19:54:250
309479932,0cyclictest15776-21awk00:09:250
309479932,0cyclictest14949-21ssh23:10:280
309479932,0cyclictest13352-21ssh00:06:340
309479932,0cyclictest12417-21cut23:49:200
309479932,0cyclictest11302-21awk21:19:250
309479930,0cyclictest415-21grep21:29:270
309479930,0cyclictest19962-21perl21:24:250
309479930,0cyclictest0-21swapper/021:39:380
29815030,0irq/29-eth0-tx-28137-21aten2.4-expect21:09:210
29805030,0irq/28-eth0-rx-0-21swapper/319:59:223
19130,0ktimersoftd/127720-21sh22:40:161
169930,0watchdog/10-21swapper/123:26:131
149930,0migration/0995-21sh23:23:480
149930,0migration/08158-21sh00:24:150
329920,0watchdog/30-21swapper/320:39:123
309479920,0cyclictest0-21swapper/022:39:120
169920,0watchdog/10-21swapper/119:19:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional