You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 00:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Mon Dec 22, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3032525655,0sleep00-21swapper/007:06:420
3048324342,0sleep20-21swapper/207:08:322
3059023929,0sleep30-21swapper/307:09:503
3052323425,0sleep10-21swapper/107:09:001
8305060,0irq/26-0000:00:5602-21latency_hist07:15:483
339960,0migration/326841-21munin-node12:01:083
339960,0migration/323677-21sh10:27:203
339960,0migration/315372-21munin-node09:46:103
307479966,0cyclictest9817-21diskmemload10:58:132
307469965,0cyclictest10973-21sed07:41:061
307459965,0cyclictest8852-21perl12:31:060
29805060,0irq/28-eth0-rx-6836-21perl08:01:033
29805060,0irq/28-eth0-rx-14107-21grep12:31:123
29805060,0irq/28-eth0-rx-0-21swapper/310:36:073
27160,0ktimersoftd/20-21swapper/211:22:402
19160,0ktimersoftd/10-21swapper/110:36:191
149960,0migration/06700-21sendmail_mailtr11:11:130
149960,0migration/030179-21sh10:14:580
149960,0migration/026203-21runrttasks11:24:060
149960,0migration/022859-21smart_nvme0n110:26:140
8305050,0irq/26-0000:00:14316-21cut09:30:493
8305050,0irq/26-0000:00:0-21swapper/309:25:493
339950,0migration/39217-21taskset07:39:323
339950,0migration/38693-21kthreadcore12:11:083
339950,0migration/3703-21fschecks_count11:11:053
339950,0migration/35234-21munin-node11:46:183
339950,0migration/3482-21kthreadcore12:26:073
339950,0migration/34012-21taskset07:56:073
339950,0migration/33159-21munin-node08:21:033
339950,0migration/331049-21kthreadcore08:16:073
339950,0migration/330349-21kthreadcore09:01:073
339950,0migration/329528-21kthreadcore07:31:073
339950,0migration/329274-21kthreadcore11:06:093
339950,0migration/328450-21munin-node11:26:063
339950,0migration/324425-21runrttasks12:37:553
339950,0migration/324243-21fschecks_time09:36:013
339950,0migration/324153-21munin-node09:51:083
339950,0migration/321604-21kthreadcore11:56:093
339950,0migration/320923-21kthreadcore11:21:073
339950,0migration/320152-21taskset08:09:233
339950,0migration/320115-21aten_r1power_en10:46:023
339950,0migration/319765-21kthreadcore11:01:083
339950,0migration/319760-21sh12:16:133
339950,0migration/314635-21kthreadcore08:26:073
339950,0migration/314586-21kthreadcore10:06:073
339950,0migration/310694-21fschecks_count09:11:033
329950,0watchdog/332629-21kthreadcore09:21:083
329950,0watchdog/323704-21kthreadcore08:56:083
329950,0watchdog/321761-21cut08:31:083
329950,0watchdog/31727-21ssh09:56:443
329950,0watchdog/315769-21kthreadcore07:21:073
307479955,0cyclictest0-21swapper/211:09:252
307479955,0cyclictest0-21swapper/210:50:392
307479955,0cyclictest0-21swapper/208:26:142
307479954,0cyclictest21583-21kthreadcore08:31:082
307479950,0cyclictest0-21swapper/212:11:222
307479950,0cyclictest0-21swapper/211:48:312
307469955,0cyclictest0-21swapper/111:27:531
307469955,0cyclictest0-21swapper/107:16:151
307469950,0cyclictest0-21swapper/112:20:131
307469950,0cyclictest0-21swapper/112:12:391
307459950,0cyclictest0-21swapper/011:57:410
307459950,0cyclictest0-21swapper/011:51:130
307459950,0cyclictest0-21swapper/011:51:130
307459950,0cyclictest0-21swapper/009:16:140
29815050,0irq/29-eth0-tx-8433-21which10:36:112
29815050,0irq/29-eth0-tx-29201-21automount12:21:372
29815050,0irq/29-eth0-tx-27718-21munin-node08:16:090
29815050,0irq/29-eth0-tx-2245-21sendmail_mailqu09:21:112
29815050,0irq/29-eth0-tx-2194-21ssh09:57:492
29815050,0irq/29-eth0-tx-20263-21ssh09:16:062
29815050,0irq/29-eth0-tx-18602-21irqstats12:36:092
29815050,0irq/29-eth0-tx-16066-21sort09:31:082
29815050,0irq/29-eth0-tx-12939-21munin-node10:06:122
29815050,0irq/29-eth0-tx-12609-21kthreadcore09:46:082
29815050,0irq/29-eth0-tx-0-21swapper/212:34:262
29815050,0irq/29-eth0-tx-0-21swapper/008:26:000
29815050,0irq/29-eth0-tx-0-21swapper/008:21:110
29815050,0irq/29-eth0-tx-0-21swapper/008:10:190
29815050,0irq/29-eth0-tx-0-21swapper/008:01:020
29815050,0irq/29-eth0-tx-0-21swapper/007:57:190
29805050,0irq/28-eth0-rx-22712-21fschecks_time10:11:023
29805050,0irq/28-eth0-rx-2035-21cut09:41:083
29805050,0irq/28-eth0-rx-11427-21kthreadcore11:51:083
29805050,0irq/28-eth0-rx-11427-21kthreadcore11:51:083
29805050,0irq/28-eth0-rx-0-21swapper/312:23:343
29805050,0irq/28-eth0-rx-0-21swapper/312:06:173
29805050,0irq/28-eth0-rx-0-21swapper/311:39:143
29805050,0irq/28-eth0-rx-0-21swapper/311:31:073
29805050,0irq/28-eth0-rx-0-21swapper/310:57:283
29805050,0irq/28-eth0-rx-0-21swapper/310:51:073
29805050,0irq/28-eth0-rx-0-21swapper/310:41:003
29805050,0irq/28-eth0-rx-0-21swapper/310:01:003
29805050,0irq/28-eth0-rx-0-21swapper/309:16:073
29805050,0irq/28-eth0-rx-0-21swapper/308:36:133
27150,0ktimersoftd/20-21swapper/210:12:302
27150,0ktimersoftd/20-21swapper/207:59:552
259950,0migration/25527-21timerwakeupswit12:26:152
259950,0migration/229283-21cpuspeed_turbos10:51:022
259950,0migration/224165-21runrttasks08:35:522
249950,0watchdog/232047-21tune2fs08:41:042
249950,0watchdog/228559-21fschecks_time08:16:042
249950,0watchdog/224932-21tr07:26:072
19150,0ktimersoftd/10-21swapper/111:06:451
179950,0migration/16598-21kthreadcore08:21:081
179950,0migration/15497-21taskset07:14:411
179950,0migration/123335-21kthreadcore08:56:081
179950,0migration/114252-21latency_hist09:30:491
149950,0migration/09225-21kthreadcore12:11:080
149950,0migration/0669-21sh12:03:390
149950,0migration/06596-21irqcore09:26:070
149950,0migration/04056-21kthreadcore11:46:110
149950,0migration/032398-21kthreadcore11:26:080
149950,0migration/032200-21kthreadcore09:21:080
149950,0migration/032073-21kthreadcore09:56:090
149950,0migration/031963-21idleruntime07:11:050
149950,0migration/031565-21cpuspeed_turbos12:26:020
149950,0migration/031190-21kthreadcore10:51:070
149950,0migration/030332-21sh09:36:260
149950,0migration/028506-21pmu-power12:21:090
149950,0migration/028269-21fschecks_time07:31:030
149950,0migration/02693-21latency_hist09:05:490
149950,0migration/026167-21kthreadcore07:51:070
149950,0migration/025580-21aten_r1power_po10:31:010
149950,0migration/025119-21gltestperf08:36:060
149950,0migration/024668-21hwlatdetect11:06:060
149950,0migration/017906-21gltestperf08:31:040
149950,0migration/017891-21kthreadcore11:36:100
149950,0migration/017783-21sh09:13:550
149950,0migration/016999-21sh11:18:290
149950,0migration/016715-21kthreadcore08:51:090
149950,0migration/015791-21kthreadcore07:21:070
149950,0migration/015351-21fschecks_time09:31:030
149950,0migration/01281-21kthreadcore10:16:070
149950,0migration/010812-21irqrtprio09:46:060
149950,0migration/010563-21sh10:40:190
339940,0migration/332713-21kthreadcore07:11:083
339940,0migration/324034-21kthreadcore11:41:083
339940,0migration/322821-21kthreadcore07:26:073
339940,0migration/319157-21kthreadcore08:51:103
329940,0watchdog/32795-21basename09:06:003
329940,0watchdog/321222-21fschecks_time08:11:043
329940,0watchdog/32071-21cat08:41:083
329940,0watchdog/30-21swapper/307:51:083
329940,0watchdog/30-21swapper/307:46:083
307479943,0cyclictest3571-21lldpd07:50:492
307479943,0cyclictest3571-21lldpd07:35:492
307479943,0cyclictest3571-21lldpd07:30:502
307479943,0cyclictest3524-21snmpd08:22:022
307479943,0cyclictest3524-21snmpd07:41:322
307479943,0cyclictest3524-21snmpd07:13:022
307479943,0cyclictest3250-21irqbalance11:35:562
307479943,0cyclictest3250-21irqbalance09:00:552
307479943,0cyclictest3250-21irqbalance08:46:252
307479943,0cyclictest17899-21awk07:46:062
307469944,0cyclictest0-21swapper/111:36:331
307469944,0cyclictest0-21swapper/110:23:051
307469943,0cyclictest7483-21awk08:46:051
307469943,0cyclictest7094-21awk08:01:051
307469943,0cyclictest5112-21awk11:31:051
307469943,0cyclictest3524-21snmpd12:22:491
307469943,0cyclictest3524-21snmpd11:46:191
307469943,0cyclictest3524-21snmpd11:25:041
307469943,0cyclictest3524-21snmpd11:13:191
307469943,0cyclictest3524-21snmpd10:51:481
307469943,0cyclictest3524-21snmpd09:01:181
307469943,0cyclictest3524-21snmpd08:31:321
307469943,0cyclictest29555-21awk09:21:061
307469943,0cyclictest26649-21awk10:31:051
307469943,0cyclictest25001-21awk07:51:041
307469943,0cyclictest20929-21sort07:26:031
307469943,0cyclictest17608-21awk10:26:071
307469943,0cyclictest14146-21awk10:06:061
307469943,0cyclictest11208-21awk09:11:051
307469943,0cyclictest10402-21awk11:16:051
307459943,0cyclictest7459-21awk08:46:050
307459943,0cyclictest3524-21snmpd07:38:320
307459943,0cyclictest23728-21sadc07:50:480
307459943,0cyclictest14440-21sadc11:00:490
29815040,0irq/29-eth0-tx-9817-21diskmemload10:16:122
29815040,0irq/29-eth0-tx-12078-21hddtemp_smartct10:41:052
29815040,0irq/29-eth0-tx-0-21swapper/212:16:062
29815040,0irq/29-eth0-tx-0-21swapper/212:06:132
29815040,0irq/29-eth0-tx-0-21swapper/211:25:522
29815040,0irq/29-eth0-tx-0-21swapper/211:10:572
29815040,0irq/29-eth0-tx-0-21swapper/211:01:182
29815040,0irq/29-eth0-tx-0-21swapper/210:25:542
29815040,0irq/29-eth0-tx-0-21swapper/210:21:452
29815040,0irq/29-eth0-tx-0-21swapper/209:52:042
29815040,0irq/29-eth0-tx-0-21swapper/209:40:572
29815040,0irq/29-eth0-tx-0-21swapper/209:36:192
29805040,0irq/28-eth0-rx-0-21swapper/311:16:083
29805040,0irq/28-eth0-rx-0-21swapper/310:31:013
29805040,0irq/28-eth0-rx-0-21swapper/310:20:513
29805040,0irq/28-eth0-rx-0-21swapper/310:16:003
29805040,0irq/28-eth0-rx-0-21swapper/308:46:013
259940,0migration/26507-21fschecks_count08:01:032
259940,0migration/228600-21kthreadcore10:31:072
259940,0migration/219983-21kthreadcore11:56:082
259940,0migration/219688-21rt-features08:51:132
259940,0migration/218245-21kthreadcore08:06:082
259940,0migration/214229-21kthreadcore11:16:082
259940,0migration/211260-21munin-node09:11:052
249940,0watchdog/28492-21hddtemp_smartct11:51:042
249940,0watchdog/28492-21hddtemp_smartct11:51:042
249940,0watchdog/28048-21kthreadcore09:26:082
249940,0watchdog/26078-21grep07:16:032
249940,0watchdog/24964-21ssh11:31:042
249940,0watchdog/23862-21if_err_eth009:06:042
249940,0watchdog/227234-21grep08:56:162
249940,0watchdog/222893-21hddtemp_smartct11:41:042
249940,0watchdog/210676-21sshd10:01:202
249940,0watchdog/20-21swapper/212:01:082
249940,0watchdog/20-21swapper/207:21:072
179940,0migration/19767-21kthreadcore12:31:101
179940,0migration/18928-21kthreadcore10:56:091
179940,0migration/132614-21kthreadcore07:31:081
179940,0migration/132598-21kthreadcore08:16:081
179940,0migration/132147-21irqcore10:16:061
179940,0migration/129542-21kthreadcore09:56:081
179940,0migration/127261-21kthreadcore08:36:091
179940,0migration/126323-21munin-node12:01:051
179940,0migration/123018-21sshd09:35:191
179940,0migration/121218-21fschecks_time08:11:041
179940,0migration/120050-21sh10:45:581
179940,0migration/119305-21kthreadcore12:36:091
179940,0migration/117158-21fschecks_time11:56:041
179940,0migration/115663-21gltestperf11:01:041
179940,0migration/112445-21kthreadcore11:51:081
179940,0migration/112445-21kthreadcore11:51:081
179940,0migration/111412-21kthreadcore09:46:071
169940,0watchdog/13082-21sh12:08:321
169940,0watchdog/125826-21ssh09:16:241
169940,0watchdog/121849-21ssh11:40:521
169940,0watchdog/118456-21cut07:46:081
169940,0watchdog/114870-21kthreadcore08:06:081
169940,0watchdog/111064-21ssh10:40:561
169940,0watchdog/10-21swapper/112:26:081
169940,0watchdog/10-21swapper/110:01:081
169940,0watchdog/10-21swapper/109:55:201
169940,0watchdog/10-21swapper/109:41:041
169940,0watchdog/10-21swapper/108:41:041
169940,0watchdog/10-21swapper/107:36:031
149940,0migration/06426-21kthreadcore11:31:070
149940,0migration/026291-21kthreadcore08:56:090
149940,0migration/023189-21kthreadcore08:11:080
149940,0migration/021214-21sh10:10:000
149940,0migration/020932-21kthreadcore09:51:070
149940,0migration/019330-21sh10:44:490
149940,0migration/01464-21irqcore09:41:070
149940,0migration/010375-21kthreadcore07:16:090
307479932,0cyclictest3571-21lldpd08:10:492
307469933,0cyclictest32324-21awk07:56:051
307459933,0cyclictest31748-21kworker/0:212:37:090
307459933,0cyclictest31748-21kworker/0:212:06:310
307459933,0cyclictest30034-21kworker/0:011:41:030
307459933,0cyclictest18084-21kworker/0:007:25:580
307459933,0cyclictest13924-21kworker/0:110:46:040
307459932,0cyclictest8417-21perl10:21:060
307459932,0cyclictest7022-21cut10:01:070
307459932,0cyclictest17790-21kthreadcore12:16:090
307459930,0cyclictest27949-21munin-node09:01:030
307459930,0cyclictest18760-21kthreadcore11:01:080
307459930,0cyclictest10410-21perl07:41:030
29815030,0irq/29-eth0-tx-0-21swapper/008:41:020
29805030,0irq/28-eth0-rx-0-21swapper/307:41:373
179930,0migration/129377-21latency09:36:061
169930,0watchdog/12796-21aten_r1power_cu09:06:001
169930,0watchdog/119520-21ntpq08:51:121
169930,0watchdog/10-21swapper/110:10:561
169930,0watchdog/10-21swapper/108:26:041
169930,0watchdog/10-21swapper/107:21:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional