You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 03:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Sun Dec 14, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1186523727,0sleep30-21swapper/319:06:033
12281993434,0cyclictest2910-21kworker/0:222:51:370
1210023228,0sleep20-21swapper/219:08:502
1206722925,0sleep00-21swapper/019:08:240
1182022824,0sleep10-21swapper/119:05:331
35160,0ktimersoftd/30-21swapper/321:51:163
339960,0migration/327625-21sh00:23:163
339960,0migration/319316-21sh22:33:263
339960,0migration/318180-21sh23:06:373
29805060,0irq/28-eth0-rx-32743-21kthreadcore21:30:223
29805060,0irq/28-eth0-rx-28152-21ssh22:02:103
29805060,0irq/28-eth0-rx-19765-21kthreadcore21:40:243
29805060,0irq/28-eth0-rx-0-21swapper/322:48:103
149960,0migration/0735-21kthreadcore20:50:250
149960,0migration/029474-21munin-node19:40:250
149960,0migration/024004-21sh23:29:390
149960,0migration/019262-21dump-pmu-power00:20:020
149960,0migration/012307-21switchtime23:55:320
122839966,0cyclictest0-21swapper/222:12:302
122839960,0cyclictest0-21swapper/221:41:462
339950,0migration/39729-21kthreadcore21:35:233
339950,0migration/36631-21gltestperf00:30:223
339950,0migration/36542-21kthreadcore23:35:253
339950,0migration/3639-21missed_timers20:05:263
339950,0migration/34471-21kthreadcore20:55:233
339950,0migration/332228-21kthreadcore20:50:243
339950,0migration/331691-21kthreadcore23:15:223
339950,0migration/331540-21switchtime22:20:323
339950,0migration/331377-21timerwakeupswit22:55:323
339950,0migration/326679-21timerwakeupswit23:10:333
339950,0migration/324133-21kthreadcore19:15:333
339950,0migration/317391-21fschecks_count21:05:213
339950,0migration/316915-21kthreadcore00:35:253
339950,0migration/316299-21taskset19:35:143
339950,0migration/316030-21taskset20:16:563
339950,0migration/314916-21kthreadcore21:55:223
339950,0migration/311928-21kthreadcore19:30:233
339950,0migration/311215-21kthreadcore22:10:233
339950,0migration/310610-21sh23:40:153
329950,0watchdog/38025-21sort20:10:243
329950,0watchdog/36696-21kthreadcore23:00:243
329950,0watchdog/3589-21cut19:45:243
329950,0watchdog/328967-21cut21:45:243
329950,0watchdog/326996-21kthreadcore19:40:243
329950,0watchdog/326215-21needreboot20:45:283
329950,0watchdog/320639-21perl23:45:203
329950,0watchdog/30-21swapper/323:56:293
29815050,0irq/29-eth0-tx-20045-21munin-node20:00:210
29815050,0irq/29-eth0-tx-1757-21munin-node19:25:250
29815050,0irq/29-eth0-tx-16307-21munin-node19:35:290
29815050,0irq/29-eth0-tx-0-21swapper/021:10:130
29815050,0irq/29-eth0-tx-0-21swapper/020:15:230
29805050,0irq/28-eth0-rx-5730-21tr21:15:253
29805050,0irq/28-eth0-rx-5146-21ssh22:07:293
29805050,0irq/28-eth0-rx-29970-21munin-node22:40:213
29805050,0irq/28-eth0-rx-28657-21ls00:25:073
29805050,0irq/28-eth0-rx-2502-21cat23:50:243
29805050,0irq/28-eth0-rx-24195-21ssh00:05:083
29805050,0irq/28-eth0-rx-24085-21aten_r1power_cu21:10:163
29805050,0irq/28-eth0-rx-16152-21fschecks_time23:25:213
29805050,0irq/28-eth0-rx-12143-21switchtime20:35:313
29805050,0irq/28-eth0-rx-0-21swapper/323:20:103
29805050,0irq/28-eth0-rx-0-21swapper/322:50:203
29805050,0irq/28-eth0-rx-0-21swapper/322:35:243
29805050,0irq/28-eth0-rx-0-21swapper/322:25:233
29805050,0irq/28-eth0-rx-0-21swapper/321:30:063
29805050,0irq/28-eth0-rx-0-21swapper/300:19:353
29805050,0irq/28-eth0-rx-0-21swapper/300:12:443
29805050,0irq/28-eth0-rx-0-21swapper/300:01:213
249950,0watchdog/222091-21kthreadcore21:25:242
179950,0migration/19378-21kthreadcore22:45:231
179950,0migration/17619-21munin-node23:35:281
179950,0migration/130724-21sh21:10:311
179950,0migration/130511-21taskset20:27:111
179950,0migration/12896-21munin-node19:25:211
179950,0migration/12894-21fschecks_count20:55:191
179950,0migration/127683-21sh23:48:001
179950,0migration/119648-21latency_hist19:15:061
179950,0migration/117855-21idleruntime21:05:221
179950,0migration/117166-21kthreadcore23:25:251
179950,0migration/116960-21kthreadcore22:30:251
179950,0migration/11547-21kthreadcore20:30:261
179950,0migration/111877-21missed_timers23:55:261
169950,0watchdog/15065-21ssh23:54:051
169950,0watchdog/122293-21kthreadcore20:20:241
149950,0migration/0958-21sh22:23:470
149950,0migration/06727-21kthreadcore23:35:250
149950,0migration/06368-21kthreadcore21:15:250
149950,0migration/04597-21kthreadcore21:50:240
149950,0migration/03786-21kthreadcore20:10:220
149950,0migration/031435-21kthreadcore23:50:230
149950,0migration/030583-21kthreadcore22:55:250
149950,0migration/029690-21fschecks_time00:25:200
149950,0migration/029636-21latency_hist22:05:060
149950,0migration/028572-21kthreadcore21:45:240
149950,0migration/028169-21gltestperf20:05:210
149950,0migration/027042-21kthreadcore00:05:250
149950,0migration/024231-21sh21:43:270
149950,0migration/023654-21latency_hist20:25:060
149950,0migration/023398-21kthreadcore20:45:260
149950,0migration/022492-21kthreadcore21:25:240
149950,0migration/020344-21fschecks_time00:20:200
149950,0migration/019763-21sh23:09:580
149950,0migration/018258-21sh23:42:270
149950,0migration/015466-21fschecks_count00:35:210
149950,0migration/012908-21taskset00:31:320
149950,0migration/010016-21sh22:28:170
122839955,0cyclictest0-21swapper/200:13:162
122839954,0cyclictest30914-21awk22:05:232
122839950,0cyclictest0-21swapper/223:43:092
122839950,0cyclictest0-21swapper/223:13:212
122839950,0cyclictest0-21swapper/221:31:312
122839950,0cyclictest0-21swapper/221:21:352
122819955,0cyclictest0-21swapper/023:30:100
122819955,0cyclictest0-21swapper/023:20:140
122819955,0cyclictest0-21swapper/023:10:180
122819955,0cyclictest0-21swapper/023:00:410
122819955,0cyclictest0-21swapper/021:31:180
122819950,0cyclictest0-21swapper/022:40:490
122819950,0cyclictest0-21swapper/022:30:530
8305040,0irq/26-0000:00:0-21swapper/222:35:062
339940,0migration/34275-21kthreadcore19:25:233
339940,0migration/325065-21kthreadcore20:00:233
339940,0migration/322701-21runrttasks22:15:373
339940,0migration/316527-21latency_hist20:20:063
339940,0migration/314532-21kthreadcore19:10:193
339940,0migration/31230-21taskset19:21:483
339940,0migration/311417-21kthreadcore21:00:233
339940,0migration/311161-21fschecks_count21:20:203
329940,0watchdog/317837-21kthreadcore19:55:243
329940,0watchdog/313189-21aten_r1power_fr20:40:163
329940,0watchdog/30-21swapper/319:50:203
29815040,0irq/29-eth0-tx-8715-21ssh23:36:422
29815040,0irq/29-eth0-tx-7017-21chrt21:15:262
29815040,0irq/29-eth0-tx-24936-21ssh00:05:212
29815040,0irq/29-eth0-tx-0-21swapper/223:23:072
29815040,0irq/29-eth0-tx-0-21swapper/223:05:232
29815040,0irq/29-eth0-tx-0-21swapper/222:46:062
29815040,0irq/29-eth0-tx-0-21swapper/200:25:242
29805040,0irq/28-eth0-rx-0-21swapper/323:30:213
259940,0migration/29690-21fschecks_count19:30:192
259940,0migration/29409-21aten_r1power_fr20:15:182
259940,0migration/29319-21sh21:51:242
259940,0migration/28998-21kthreadcore23:55:252
259940,0migration/27935-21kthreadcore20:35:232
259940,0migration/227150-21kthreadcore21:45:232
259940,0migration/218815-21kthreadcore00:35:262
249940,0watchdog/225061-21ssh22:20:122
249940,0watchdog/223176-21kthreadcore00:20:252
249940,0watchdog/222910-21kthreadcore23:45:242
249940,0watchdog/219925-21kthreadcore22:15:242
249940,0watchdog/217931-21ssh00:17:132
249940,0watchdog/212550-21if_eth1.321:55:202
249940,0watchdog/20-21swapper/219:45:242
249940,0watchdog/20-21swapper/219:25:242
179940,0migration/13453-21kthreadcore23:00:231
179940,0migration/129788-21kthreadcore20:50:241
179940,0migration/127500-21kthreadcore00:05:251
179940,0migration/126554-21sh00:20:591
179940,0migration/126348-21idleruntime21:45:211
179940,0migration/125075-21fschecks_count23:30:181
179940,0migration/123906-21df_abs19:40:181
179940,0migration/11812-21kthreadcore22:05:251
179940,0migration/113587-21sh21:35:551
179940,0migration/113120-21kthreadcore23:40:241
169940,0watchdog/18518-21munin-node21:50:281
169940,0watchdog/14847-21grep00:10:251
169940,0watchdog/131962-21ssh22:56:281
169940,0watchdog/130463-21munin-node21:30:201
169940,0watchdog/125547-21diskstats22:20:201
169940,0watchdog/124599-21kthreadcore22:35:241
169940,0watchdog/120813-21kthreadcore21:40:241
169940,0watchdog/115069-21aten2.4-expect00:00:171
169940,0watchdog/114024-21cut23:05:241
169940,0watchdog/10-21swapper/122:15:241
169940,0watchdog/10-21swapper/121:55:201
169940,0watchdog/10-21swapper/121:15:081
169940,0watchdog/10-21swapper/119:35:201
149940,0migration/07404-21fschecks_time22:10:180
149940,0migration/05514-21sh23:19:220
149940,0migration/015396-21kthreadcore19:55:230
122839944,0cyclictest0-21swapper/221:05:302
122839943,0cyclictest6864-21awk00:30:242
122839943,0cyclictest3571-21lldpd23:00:202
122839943,0cyclictest3571-21lldpd22:55:192
122839943,0cyclictest3571-21lldpd22:25:492
122839943,0cyclictest3571-21lldpd21:10:182
122839943,0cyclictest3571-21lldpd20:58:482
122839943,0cyclictest3571-21lldpd19:40:152
122839943,0cyclictest3571-21lldpd19:10:142
122839943,0cyclictest3524-21snmpd23:50:422
122839943,0cyclictest3524-21snmpd23:25:572
122839943,0cyclictest3524-21snmpd22:51:262
122839943,0cyclictest3524-21snmpd22:30:112
122839943,0cyclictest3524-21snmpd21:36:262
122839943,0cyclictest3524-21snmpd21:04:552
122839943,0cyclictest3524-21snmpd20:50:252
122839943,0cyclictest3524-21snmpd20:45:102
122839943,0cyclictest3524-21snmpd20:32:552
122839943,0cyclictest3524-21snmpd20:26:252
122839943,0cyclictest3524-21snmpd20:10:252
122839943,0cyclictest3524-21snmpd19:54:242
122839943,0cyclictest3524-21snmpd19:20:542
122839943,0cyclictest3250-21irqbalance20:20:512
122839943,0cyclictest30633-21awk22:40:242
122839943,0cyclictest30466-21awk23:15:202
122839943,0cyclictest28303-21awk20:05:222
122839943,0cyclictest25605-21awk23:30:202
122839943,0cyclictest21755-21awk22:00:222
122839943,0cyclictest17459-21awk19:35:232
122839943,0cyclictest16151-21awk00:00:212
122839943,0cyclictest14004-21awk19:55:212
122829944,0cyclictest0-21swapper/100:15:451
122829943,0cyclictest3524-21snmpd22:29:261
122829943,0cyclictest3524-21snmpd20:45:251
122829943,0cyclictest3524-21snmpd20:36:251
122829943,0cyclictest3250-21irqbalance23:20:221
122829943,0cyclictest3250-21irqbalance23:15:121
122829943,0cyclictest3250-21irqbalance23:10:221
122829943,0cyclictest3250-21irqbalance22:51:121
122829943,0cyclictest3250-21irqbalance22:40:221
122829943,0cyclictest3250-21irqbalance21:25:211
122829943,0cyclictest3250-21irqbalance21:20:411
122829943,0cyclictest3250-21irqbalance21:00:411
122829943,0cyclictest3250-21irqbalance20:17:311
122829943,0cyclictest3250-21irqbalance20:10:411
122829943,0cyclictest3250-21irqbalance20:06:211
122829943,0cyclictest3250-21irqbalance20:01:511
122829943,0cyclictest3250-21irqbalance00:35:221
122829943,0cyclictest3250-21irqbalance00:31:021
122829943,0cyclictest29899-21awk00:25:221
122829943,0cyclictest13792-21tail19:55:191
122829943,0cyclictest10233-21awk19:30:211
122819944,0cyclictest0-21swapper/021:21:220
122819943,0cyclictest3524-21snmpd00:13:430
122819943,0cyclictest3250-21irqbalance23:45:320
122819943,0cyclictest3250-21irqbalance22:36:420
122819943,0cyclictest3250-21irqbalance22:15:220
122819943,0cyclictest3250-21irqbalance22:05:220
122819943,0cyclictest3250-21irqbalance21:56:420
122819943,0cyclictest3250-21irqbalance21:36:420
122819943,0cyclictest3250-21irqbalance20:36:310
122819943,0cyclictest3250-21irqbalance19:50:510
122819943,0cyclictest3250-21irqbalance19:45:110
122819943,0cyclictest3250-21irqbalance19:30:410
122819943,0cyclictest3250-21irqbalance19:17:310
122819943,0cyclictest3250-21irqbalance19:10:510
122819943,0cyclictest31010-21munin-node20:30:340
122819943,0cyclictest28012-21awk19:20:200
122819943,0cyclictest24885-21awk20:25:190
122819943,0cyclictest14215-21awk20:40:200
29815030,0irq/29-eth0-tx-0-21swapper/021:05:270
29815030,0irq/29-eth0-tx-0-21swapper/020:55:170
29805030,0irq/28-eth0-rx-0-21swapper/320:30:303
29805030,0irq/28-eth0-rx-0-21swapper/320:25:153
249930,0watchdog/20-21swapper/220:40:162
169930,0watchdog/124950-21ssh22:00:241
169930,0watchdog/10-21swapper/120:40:281
169930,0watchdog/10-21swapper/119:50:201
169930,0watchdog/10-21swapper/119:10:191
122839932,0cyclictest25939-21sensors19:15:372
122839932,0cyclictest21198-21awk20:00:212
122829932,0cyclictest6399-21sadc22:10:061
122829932,0cyclictest31823-21awk19:45:221
122829932,0cyclictest1101-21users19:20:301
122819933,0cyclictest2910-21kworker/0:222:45:280
122819933,0cyclictest2591-21kworker/0:000:00:150
122819933,0cyclictest17519-21kworker/0:121:00:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional