You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 10:20
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Mon Dec 22, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1688823626,0sleep30-21swapper/319:08:083
1699623526,0sleep20-21swapper/219:09:232
1687622925,0sleep10-21swapper/119:07:581
1020322824,0sleep00-21swapper/019:05:570
29805070,0irq/28-eth0-rx-25163-21iostat_ios21:26:063
339960,0migration/33423-21sh21:12:253
339960,0migration/329164-21sh22:39:243
339960,0migration/323843-21munin-node22:01:023
339960,0migration/323756-21runrttasks00:23:483
29815060,0irq/29-eth0-tx-31993-21munin-node20:05:560
29805060,0irq/28-eth0-rx-31768-21ls20:05:463
171899966,0cyclictest0-21swapper/223:12:092
171899960,0cyclictest0-21swapper/222:49:182
171899960,0cyclictest0-21swapper/200:26:442
171879960,0cyclictest22670-21ssh22:51:430
149960,0migration/027366-21kthreadcore23:31:060
149960,0migration/023571-21ntp_offset19:56:080
8305050,0irq/26-0000:00:28116-21sed21:45:472
8305050,0irq/26-0000:00:24396-21perf19:15:462
339950,0migration/38030-21unixbench_singl23:01:133
339950,0migration/3560-21sh23:51:143
339950,0migration/35203-21taskset21:15:543
339950,0migration/34577-21irqrtprio19:46:033
339950,0migration/327977-21taskset20:22:333
339950,0migration/32666-21if_eth1.200:31:043
339950,0migration/323712-21switchtime19:11:113
339950,0migration/323268-21kthreadcore21:06:053
339950,0migration/320296-21gltestperf23:11:013
339950,0migration/31700-21kthreadcore23:36:083
339950,0migration/316566-21kthreadcore22:31:053
339950,0migration/314630-21taskset21:20:553
339950,0migration/314172-21aten_r1power_vo23:25:593
339950,0migration/31097-21irqrtprio20:06:023
339950,0migration/310686-21kthreadcore20:56:063
329950,0watchdog/3900-21ssh22:21:383
329950,0watchdog/33774-21ksmtuned23:23:423
329950,0watchdog/320324-21swap20:16:133
329950,0watchdog/313134-21python22:46:103
329950,0watchdog/312743-21fschecks_count00:01:023
329950,0watchdog/30-21swapper/323:05:503
329950,0watchdog/30-21swapper/300:26:023
29815050,0irq/29-eth0-tx-31840-21munin-node22:06:112
29815050,0irq/29-eth0-tx-0-21swapper/221:57:382
29815050,0irq/29-eth0-tx-0-21swapper/020:50:590
29815050,0irq/29-eth0-tx-0-21swapper/020:36:050
29815050,0irq/29-eth0-tx-0-21swapper/020:21:080
29815050,0irq/29-eth0-tx-0-21swapper/019:29:580
29805050,0irq/28-eth0-rx-8115-21if_eth1.200:16:053
29805050,0irq/28-eth0-rx-6147-21sensors20:51:103
29805050,0irq/28-eth0-rx-5047-21cpu21:51:003
29805050,0irq/28-eth0-rx-5047-21cpu21:51:003
29805050,0irq/28-eth0-rx-4121-21python22:41:113
29805050,0irq/28-eth0-rx-22625-21ssh22:51:373
29805050,0irq/28-eth0-rx-17973-21kthreadcore19:31:043
29805050,0irq/28-eth0-rx-12769-21munin-node20:11:103
29805050,0irq/28-eth0-rx-0-21swapper/323:58:133
29805050,0irq/28-eth0-rx-0-21swapper/323:46:083
29805050,0irq/28-eth0-rx-0-21swapper/322:55:483
29805050,0irq/28-eth0-rx-0-21swapper/322:30:343
29805050,0irq/28-eth0-rx-0-21swapper/322:19:363
29805050,0irq/28-eth0-rx-0-21swapper/322:11:013
29805050,0irq/28-eth0-rx-0-21swapper/322:06:063
29805050,0irq/28-eth0-rx-0-21swapper/321:56:043
29805050,0irq/28-eth0-rx-0-21swapper/321:36:243
29805050,0irq/28-eth0-rx-0-21swapper/300:10:573
27150,0ktimersoftd/20-21swapper/221:32:502
27150,0ktimersoftd/20-21swapper/221:00:492
259950,0migration/24797-21sh21:15:322
249950,0watchdog/28854-21kthreadcore19:26:052
249950,0watchdog/216579-21fschecks_time00:21:022
19150,0ktimersoftd/10-21swapper/122:09:091
19150,0ktimersoftd/10-21swapper/122:01:331
171899955,0cyclictest0-21swapper/223:38:292
171899955,0cyclictest0-21swapper/222:35:542
171899955,0cyclictest0-21swapper/221:20:412
171899955,0cyclictest0-21swapper/200:04:392
171899950,0cyclictest0-21swapper/223:35:292
171899950,0cyclictest0-21swapper/222:10:492
171889955,0cyclictest0-21swapper/122:17:471
171889955,0cyclictest0-21swapper/121:30:581
171889954,0cyclictest14336-21sadc21:20:471
171889950,0cyclictest0-21swapper/123:35:591
171889950,0cyclictest0-21swapper/123:24:281
171879950,0cyclictest28535-21diskmemload00:35:560
179950,0migration/17325-21mdstat23:01:061
179950,0migration/17248-21aten_r1power_fr20:56:001
179950,0migration/16340-21taskset00:15:091
179950,0migration/16175-21sh22:45:071
179950,0migration/16089-21timerwakeupswit20:06:101
179950,0migration/128936-21fschecks_count20:26:001
179950,0migration/124404-21munin-node23:46:161
179950,0migration/117855-21kthreadcore21:21:031
179950,0migration/116936-21munin-node19:51:121
179950,0migration/116333-21munin-node00:21:011
179950,0migration/115245-21if_eth120:16:041
169950,0watchdog/121297-21munin-node20:21:051
149950,0migration/09246-21kthreadcore20:56:060
149950,0migration/0917-21taskset19:21:040
149950,0migration/09150-21needreboot22:26:080
149950,0migration/08510-21kthreadcore19:46:050
149950,0migration/07775-21kthreadcore21:51:050
149950,0migration/07775-21kthreadcore21:51:050
149950,0migration/0535-21kthreadcore21:46:050
149950,0migration/04464-21munin-node22:41:130
149950,0migration/04462-21kthreadcore23:36:080
149950,0migration/04034-21kthreadcore00:31:050
149950,0migration/03758-21kthreadcore22:06:060
149950,0migration/029977-21kthreadcore20:46:060
149950,0migration/029883-21kthreadcore19:16:060
149950,0migration/029424-21kthreadcore22:56:070
149950,0migration/02795-21runrttasks19:44:550
149950,0migration/027629-21unixbench_singl19:36:120
149950,0migration/027489-21idleruntime22:21:030
149950,0migration/025301-21fschecks_count20:00:570
149950,0migration/021015-21runrttasks22:35:520
149950,0migration/020973-21taskset20:19:510
149950,0migration/020196-21fschecks_time23:11:010
149950,0migration/019540-21kthreadcore20:41:050
149950,0migration/019270-21kthreadcore21:01:060
149950,0migration/017391-21latency_hist22:15:470
149950,0migration/016795-21kthreadcore00:01:070
149950,0migration/016298-21sh23:43:220
149950,0migration/015963-21aten_r1power_en00:21:000
149950,0migration/015462-21sh00:20:280
149950,0migration/011655-21kthreadcore20:11:090
149950,0migration/010591-21cpuspeed_turbos23:05:590
8305040,0irq/26-0000:00:0-21swapper/223:26:052
8305040,0irq/26-0000:00:0-21swapper/220:00:462
339940,0migration/38653-21sh23:40:553
339940,0migration/325995-21irqcore19:16:043
339940,0migration/318023-21aten_r1power_vo20:41:003
339940,0migration/312687-21kthreadcore00:36:053
339940,0migration/311478-21fschecks_time20:36:023
339940,0migration/311016-21kthreadcore19:26:063
329940,0watchdog/329381-21munin-node19:41:013
329940,0watchdog/32934-21kthreadcore19:21:053
329940,0watchdog/320429-21munin-node21:41:053
329940,0watchdog/30-21swapper/323:31:023
329940,0watchdog/30-21swapper/321:45:533
329940,0watchdog/30-21swapper/321:31:053
329940,0watchdog/30-21swapper/320:30:573
29815040,0irq/29-eth0-tx-4095-21sensors_temp22:41:092
29815040,0irq/29-eth0-tx-2937-21kthreadcore23:01:042
29815040,0irq/29-eth0-tx-10875-21kthreadcore23:21:052
29815040,0irq/29-eth0-tx-0-21swapper/223:55:572
29815040,0irq/29-eth0-tx-0-21swapper/223:48:062
29815040,0irq/29-eth0-tx-0-21swapper/222:55:482
29815040,0irq/29-eth0-tx-0-21swapper/222:51:002
29815040,0irq/29-eth0-tx-0-21swapper/222:20:572
29815040,0irq/29-eth0-tx-0-21swapper/222:15:532
29815040,0irq/29-eth0-tx-0-21swapper/221:50:562
29815040,0irq/29-eth0-tx-0-21swapper/221:50:562
29815040,0irq/29-eth0-tx-0-21swapper/221:35:552
29815040,0irq/29-eth0-tx-0-21swapper/221:20:592
29815040,0irq/29-eth0-tx-0-21swapper/200:16:422
29815040,0irq/29-eth0-tx-0-21swapper/021:10:490
29805040,0irq/28-eth0-rx-0-21swapper/323:16:083
29805040,0irq/28-eth0-rx-0-21swapper/321:01:053
29805040,0irq/28-eth0-rx-0-21swapper/300:06:103
259940,0migration/23929-21fschecks_time19:46:002
259940,0migration/232237-21kthreadcore23:16:042
259940,0migration/23160-21latency_hist20:30:462
259940,0migration/226705-21kthreadcore00:06:062
259940,0migration/224012-21kthreadcore19:36:042
259940,0migration/222220-21kthreadcore20:41:062
259940,0migration/221988-21munin-node21:06:012
259940,0migration/216094-21kthreadcore20:36:072
249940,0watchdog/29310-21fschecks_count23:41:022
249940,0watchdog/25617-21kthreadcore00:31:062
249940,0watchdog/23490-21egrep20:30:572
249940,0watchdog/228535-21diskmemload00:12:062
249940,0watchdog/219761-21kthreadcore19:56:052
249940,0watchdog/216362-21missed_timers23:06:062
249940,0watchdog/20-21swapper/222:00:532
171899943,0cyclictest8320-21awk20:56:042
171899943,0cyclictest8076-21awk20:11:072
171899943,0cyclictest3571-21lldpd23:51:132
171899943,0cyclictest3571-21lldpd20:07:402
171899943,0cyclictest3571-21lldpd19:51:402
171899943,0cyclictest3524-21snmpd22:30:572
171899943,0cyclictest3524-21snmpd20:54:262
171899943,0cyclictest3524-21snmpd20:46:412
171899943,0cyclictest3524-21snmpd19:33:562
171899943,0cyclictest3524-21snmpd19:20:562
171899943,0cyclictest3250-21irqbalance00:35:532
171899943,0cyclictest29367-21awk19:41:012
171899943,0cyclictest25669-21awk19:16:032
171899943,0cyclictest25037-21awk21:26:052
171899943,0cyclictest21062-21sadc20:20:462
171899943,0cyclictest20412-21awk21:41:052
171899942,0cyclictest3571-21lldpd22:26:112
171889943,0cyclictest5972-21awk21:51:031
171889943,0cyclictest5972-21awk21:51:031
171889943,0cyclictest4227-21awk19:46:021
171889943,0cyclictest3524-21snmpd20:14:411
171889943,0cyclictest29421-21awk19:41:011
171889943,0cyclictest25949-21awk20:01:001
171889943,0cyclictest16959-21awk22:51:021
171889943,0cyclictest11703-21awk20:36:041
171889943,0cyclictest11303-21awk21:36:061
171879944,0cyclictest0-21swapper/000:28:030
171879943,0cyclictest3524-21snmpd23:49:570
171879943,0cyclictest3524-21snmpd23:16:120
171879943,0cyclictest3524-21snmpd21:08:560
171879943,0cyclictest27928-21unixbench_multi00:06:140
171879943,0cyclictest27828-21awk23:51:050
171879943,0cyclictest22750-21sadc23:30:460
171879943,0cyclictest17427-21sort19:31:040
171879943,0cyclictest11549-21awk19:51:020
171879943,0cyclictest11165-21gltestperf21:36:040
171879940,0cyclictest0-21swapper/020:26:000
179940,0migration/19483-21kthreadcore22:46:051
179940,0migration/17541-21fschecks_count00:16:031
179940,0migration/17290-21kthreadcore22:26:061
179940,0migration/16256-21kthreadcore23:56:051
179940,0migration/16170-21kthreadcore20:31:031
179940,0migration/1441-21kthreadcore23:16:051
179940,0migration/13256-21kthreadcore00:31:051
179940,0migration/127578-21kthreadcore19:16:051
179940,0migration/126895-21kthreadcore20:46:051
179940,0migration/122093-21fschecks_time00:06:021
179940,0migration/121932-21fschecks_time22:36:011
179940,0migration/120759-21sh21:56:401
179940,0migration/116555-21runrttasks22:14:161
179940,0migration/116493-21missed_timers00:36:071
179940,0migration/113931-21runrttasks19:30:581
169940,0watchdog/16314-21runrttasks23:25:501
169940,0watchdog/13936-21rm21:49:211
169940,0watchdog/131147-21kthreadcore22:21:061
169940,0watchdog/129382-21munin-node23:31:101
169940,0watchdog/123278-21missed_timers19:11:051
169940,0watchdog/120809-21kthreadcore20:41:051
169940,0watchdog/116716-21kthreadcore21:01:051
169940,0watchdog/10-21swapper/123:06:141
169940,0watchdog/10-21swapper/122:31:061
169940,0watchdog/10-21swapper/121:10:491
149940,0migration/0846-21sh21:30:590
149940,0migration/04882-21sh00:11:440
149940,0migration/023813-21timerwakeupswit19:11:110
149940,0migration/022215-21kthreadcore21:41:070
149940,0migration/02044-21fschecks_time23:01:010
149940,0migration/018202-21kthreadcore21:56:040
329930,0watchdog/30-21swapper/319:56:053
329930,0watchdog/30-21swapper/319:36:333
29805030,0irq/28-eth0-rx-0-21swapper/320:45:593
29805030,0irq/28-eth0-rx-0-21swapper/320:25:583
29805030,0irq/28-eth0-rx-0-21swapper/319:51:003
249930,0watchdog/20-21swapper/220:21:052
171889932,0cyclictest925-21awk20:51:041
171889932,0cyclictest7871-21awk19:26:031
171889932,0cyclictest3524-21snmpd22:56:021
171889932,0cyclictest25569-21sensors21:41:111
171889932,0cyclictest22787-21irqcore21:06:051
171889932,0cyclictest22186-21awk19:36:021
171889932,0cyclictest20499-21awk23:11:051
171879933,0cyclictest5890-21kworker/0:122:30:580
171879933,0cyclictest5890-21kworker/0:122:10:500
171879933,0cyclictest26042-21kworker/0:121:26:040
171879933,0cyclictest26042-21kworker/0:121:20:480
171879933,0cyclictest26042-21kworker/0:121:16:000
171879933,0cyclictest22089-21kworker/0:220:30:530
171879933,0cyclictest22049-21kworker/0:222:46:060
171879933,0cyclictest1826-21kworker/0:022:00:580
171879933,0cyclictest11380-21kworker/0:123:21:220
171879932,0cyclictest4473-21awk23:56:030
169930,0watchdog/124354-21munin-node21:26:011
169930,0watchdog/10-21swapper/123:51:021
169930,0watchdog/10-21swapper/123:41:021
169930,0watchdog/10-21swapper/119:56:051
169930,0watchdog/10-21swapper/119:21:011
169930,0watchdog/10-21swapper/100:26:021
169930,0watchdog/10-21swapper/100:01:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional