You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 05:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Wed Dec 24, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
625924132,0sleep20-21swapper/219:07:542
643624030,0sleep30-21swapper/319:09:553
652823624,0sleep00-21swapper/019:10:560
627123526,0sleep10-21swapper/119:08:011
65989999,0cyclictest83050irq/26-0000:00:20:56:102
8305060,0irq/26-0000:00:10153-21crond00:05:572
65969960,0cyclictest0-21swapper/022:58:270
339960,0migration/3608-21switchtime21:51:233
339960,0migration/332354-21kthreadcore21:16:153
339960,0migration/329571-21sh21:31:453
339960,0migration/325338-21sh21:48:363
29805060,0irq/28-eth0-rx-6024-21python00:36:183
29805060,0irq/28-eth0-rx-19278-21irqstats23:16:143
19160,0ktimersoftd/10-21swapper/123:32:521
8305050,0irq/26-0000:00:1150-21if_eth1.320:36:132
8305050,0irq/26-0000:00:0-21swapper/223:05:572
65989950,0cyclictest0-21swapper/223:47:582
65989950,0cyclictest0-21swapper/221:20:432
65989950,0cyclictest0-21swapper/220:06:182
65979955,0cyclictest18052-21diskmemload22:23:581
65979955,0cyclictest0-21swapper/123:55:251
65979955,0cyclictest0-21swapper/121:26:161
65979955,0cyclictest0-21swapper/100:15:071
65979950,0cyclictest0-21swapper/122:13:151
65979950,0cyclictest0-21swapper/100:08:111
65969955,0cyclictest9714-21kworker/0:223:54:100
65969955,0cyclictest24719-21kworker/0:221:41:100
65969955,0cyclictest24719-21kworker/0:221:31:160
65969955,0cyclictest24719-21kworker/0:219:46:190
65969955,0cyclictest16862-21kworker/0:123:06:380
65969955,0cyclictest16862-21kworker/0:122:55:280
65969955,0cyclictest16862-21kworker/0:122:43:560
65969950,0cyclictest0-21swapper/022:31:480
65969950,0cyclictest0-21swapper/020:36:140
65969950,0cyclictest0-21swapper/000:37:460
65969950,0cyclictest0-21swapper/000:16:200
65969950,0cyclictest0-21swapper/000:08:170
339950,0migration/38911-21sh00:03:043
339950,0migration/38075-21fschecks_time20:41:133
339950,0migration/3664-21kthreadcore19:26:163
339950,0migration/34932-21sh22:12:003
339950,0migration/34390-21munin-node21:56:133
339950,0migration/33703-21taskset00:18:313
339950,0migration/327793-21kthreadcore22:26:153
339950,0migration/324082-21munin-node23:01:153
339950,0migration/322999-21kthreadcore22:41:153
339950,0migration/321239-21kthreadcore20:26:153
339950,0migration/31975-21fschecks_count22:31:113
339950,0migration/318559-21kthreadcore20:46:143
339950,0migration/316301-21kthreadcore00:06:183
339950,0migration/315132-21kthreadcore20:21:173
339950,0migration/315120-21kthreadcore21:26:143
339950,0migration/314618-21fschecks_count19:16:113
339950,0migration/312545-21kthreadcore22:36:163
339950,0migration/312199-21kthreadcore19:56:153
339950,0migration/311012-21kthreadcore23:11:163
339950,0migration/310312-21unixbench_multi21:01:273
329950,0watchdog/331513-21ssh00:35:323
329950,0watchdog/329648-21ssh23:40:203
329950,0watchdog/30-21swapper/320:11:193
329950,0watchdog/30-21swapper/300:26:163
329950,0watchdog/30-21swapper/300:11:163
29815050,0irq/29-eth0-tx-3999-21ntp_offset23:41:182
29815050,0irq/29-eth0-tx-31831-21munin-node21:36:132
29815050,0irq/29-eth0-tx-31169-21cat00:16:142
29815050,0irq/29-eth0-tx-29441-21python23:01:242
29815050,0irq/29-eth0-tx-25978-21kthreadcore22:06:172
29815050,0irq/29-eth0-tx-24314-21kthreadcore21:31:162
29815050,0irq/29-eth0-tx-22823-21kthreadcore22:41:152
29815050,0irq/29-eth0-tx-19423-21munin-node00:11:132
29815050,0irq/29-eth0-tx-16883-21ssh23:14:432
29815050,0irq/29-eth0-tx-14514-21munin-node20:46:080
29815050,0irq/29-eth0-tx-14380-21ssh22:18:002
29815050,0irq/29-eth0-tx-12535-21ssh23:31:102
29815050,0irq/29-eth0-tx-0-21swapper/223:39:592
29815050,0irq/29-eth0-tx-0-21swapper/222:03:322
29815050,0irq/29-eth0-tx-0-21swapper/221:49:562
29815050,0irq/29-eth0-tx-0-21swapper/021:11:090
29815050,0irq/29-eth0-tx-0-21swapper/020:51:090
29815050,0irq/29-eth0-tx-0-21swapper/020:22:210
29815050,0irq/29-eth0-tx-0-21swapper/020:11:110
29815050,0irq/29-eth0-tx-0-21swapper/019:57:170
29805050,0irq/28-eth0-rx-9841-21cut19:11:153
29805050,0irq/28-eth0-rx-7133-21kthreadcore22:51:143
29805050,0irq/28-eth0-rx-6668-21kthreadcore00:21:153
29805050,0irq/28-eth0-rx-5632-21irqrtprio21:21:133
29805050,0irq/28-eth0-rx-28517-21kthreadcore23:56:183
29805050,0irq/28-eth0-rx-27963-21cut23:21:143
29805050,0irq/28-eth0-rx-18699-21tr22:21:143
29805050,0irq/28-eth0-rx-17806-21kthreadcore22:01:153
29805050,0irq/28-eth0-rx-17791-21kthreadcore20:01:163
29805050,0irq/28-eth0-rx-16371-21fschecks_count23:51:113
29805050,0irq/28-eth0-rx-1291-21kthreadcore21:36:143
29805050,0irq/28-eth0-rx-10753-21kthreadcore22:16:163
29805050,0irq/28-eth0-rx-0-21swapper/323:48:113
29805050,0irq/28-eth0-rx-0-21swapper/323:41:103
29805050,0irq/28-eth0-rx-0-21swapper/323:32:033
29805050,0irq/28-eth0-rx-0-21swapper/323:27:303
29805050,0irq/28-eth0-rx-0-21swapper/322:56:153
29805050,0irq/28-eth0-rx-0-21swapper/322:09:063
29805050,0irq/28-eth0-rx-0-21swapper/321:40:593
29805050,0irq/28-eth0-rx-0-21swapper/320:16:173
29805050,0irq/28-eth0-rx-0-21swapper/319:36:083
259950,0migration/29472-21sh22:34:092
259950,0migration/28908-21sh21:41:092
259950,0migration/22564-21sh21:55:112
259950,0migration/224647-21taskset20:27:122
259950,0migration/214196-21sh22:56:082
259950,0migration/211785-21sh21:23:092
259950,0migration/211099-21munin-node22:51:152
19150,0ktimersoftd/10-21swapper/123:42:011
19150,0ktimersoftd/10-21swapper/123:18:121
19150,0ktimersoftd/10-21swapper/100:38:271
179950,0migration/1363-21hwlatdetect21:36:111
179950,0migration/132248-21sh23:58:051
179950,0migration/132206-21aten_r1power_cu19:51:091
179950,0migration/131787-21kthreadcore00:16:141
179950,0migration/130250-21sh23:02:271
179950,0migration/121900-21fschecks_count23:36:121
179950,0migration/121641-21df_abs19:21:081
179950,0migration/120744-21irqcore22:41:141
179950,0migration/120654-21taskset19:16:541
179950,0migration/12033-21missed_timers22:46:171
179950,0migration/118582-21cpuspeed_turbos21:11:101
179950,0migration/118105-21latency_hist21:10:561
179950,0migration/116850-21kthreadcore20:46:131
179950,0migration/113255-21gltestperf22:01:111
179950,0migration/112799-21kthreadcore19:36:131
169950,0watchdog/12247-21tr20:11:191
169950,0watchdog/112662-21grep22:36:161
149950,0migration/08816-21kthreadcore21:56:160
149950,0migration/08594-21latency_hist23:10:570
149950,0migration/08357-21kthreadcore21:01:190
149950,0migration/07554-21kthreadcore23:26:140
149950,0migration/04783-21irqrtprio19:31:130
149950,0migration/030831-21kthreadcore21:51:160
149950,0migration/030012-21sh00:32:000
149950,0migration/027660-21kthreadcore20:31:150
149950,0migration/027519-21taskset00:14:420
149950,0migration/027370-21runrttasks21:16:080
149950,0migration/027356-21kthreadcore23:56:170
149950,0migration/027069-21kthreadcore23:01:180
149950,0migration/023767-21kthreadcore22:06:160
149950,0migration/022245-21kthreadcore21:46:150
149950,0migration/018554-21fschecks_time23:16:110
149950,0migration/018377-21fschecks_count19:41:080
149950,0migration/01783-21kthreadcore20:56:120
149950,0migration/017551-21kthreadcore22:21:140
149950,0migration/016860-21kthreadcore20:01:150
149950,0migration/015613-21kthreadcore22:36:160
149950,0migration/015044-21kthreadcore23:31:150
149950,0migration/013913-21munin-node21:26:100
149950,0migration/012995-21sh00:24:090
149950,0migration/010451-21taskset20:19:300
8305040,0irq/26-0000:00:0-21swapper/222:45:572
65989943,0cyclictest831-21awk19:51:142
65989943,0cyclictest3571-21lldpd19:40:542
65989943,0cyclictest3524-21snmpd19:48:282
65989943,0cyclictest26015-21awk23:56:162
65989943,0cyclictest16976-21awk23:51:132
65979944,0cyclictest0-21swapper/122:53:361
65979943,0cyclictest8037-21awk22:16:131
65979943,0cyclictest5346-21awk21:21:121
65979943,0cyclictest3571-21lldpd23:21:291
65979943,0cyclictest3524-21snmpd23:26:001
65979943,0cyclictest3524-21snmpd20:43:581
65979943,0cyclictest3524-21snmpd20:33:131
65979943,0cyclictest3524-21snmpd20:20:581
65979943,0cyclictest3524-21snmpd20:15:581
65979943,0cyclictest3524-21snmpd00:35:011
65979943,0cyclictest15400-21awk20:01:131
65979943,0cyclictest1172-21sadc22:30:561
65969944,0cyclictest9714-21kworker/0:223:41:140
65969944,0cyclictest9714-21kworker/0:223:39:100
65969944,0cyclictest5782-21kworker/0:119:21:130
65969944,0cyclictest5782-21kworker/0:119:15:370
65969944,0cyclictest24719-21kworker/0:221:22:580
65969944,0cyclictest16862-21kworker/0:122:50:480
65969944,0cyclictest0-21swapper/022:16:490
65969944,0cyclictest0-21swapper/021:40:050
65969943,0cyclictest810-21awk19:51:130
65969943,0cyclictest3250-21irqbalance23:21:130
65969943,0cyclictest3250-21irqbalance20:26:030
65969943,0cyclictest3250-21irqbalance20:06:320
65969943,0cyclictest2485-21awk00:01:150
65969943,0cyclictest15156-21awk00:26:140
339940,0migration/325039-21diskmemload21:11:323
339940,0migration/321733-21fschecks_count19:21:093
329940,0watchdog/34045-21kthreadcore20:36:153
329940,0watchdog/332021-21kthreadcore22:46:163
329940,0watchdog/30-21swapper/320:51:153
329940,0watchdog/30-21swapper/320:31:153
329940,0watchdog/30-21swapper/319:46:193
29815040,0irq/29-eth0-tx-31965-21ssh00:36:062
29815040,0irq/29-eth0-tx-23027-21ssh00:31:072
29815040,0irq/29-eth0-tx-18323-21munin-node21:11:262
29815040,0irq/29-eth0-tx-10487-21ssh22:35:592
29815040,0irq/29-eth0-tx-0-21swapper/223:26:152
29815040,0irq/29-eth0-tx-0-21swapper/222:26:202
29815040,0irq/29-eth0-tx-0-21swapper/221:56:272
29815040,0irq/29-eth0-tx-0-21swapper/200:26:072
29815040,0irq/29-eth0-tx-0-21swapper/200:06:052
29815040,0irq/29-eth0-tx-0-21swapper/019:26:100
29805040,0irq/28-eth0-rx-0-21swapper/323:06:133
29805040,0irq/28-eth0-rx-0-21swapper/321:06:093
259940,0migration/227925-21irqcore23:21:142
259940,0migration/222031-21fschecks_time19:21:092
259940,0migration/217581-21latency_hist23:15:572
259940,0migration/215811-21kthreadcore21:26:142
259940,0migration/212209-21kthreadcore20:41:172
259940,0migration/211825-21idleruntime20:21:142
259940,0migration/210525-21runrttasks20:20:242
249940,0watchdog/29154-21grep21:01:192
249940,0watchdog/26314-21runrttasks19:41:072
249940,0watchdog/228863-21munin-node19:26:112
249940,0watchdog/226229-21tr20:51:152
249940,0watchdog/216718-21kthreadcore21:06:152
249940,0watchdog/216503-21kthreadcore19:16:152
249940,0watchdog/214776-21fschecks_count20:01:112
249940,0watchdog/211443-21python00:21:202
249940,0watchdog/20-21swapper/222:21:082
249940,0watchdog/20-21swapper/222:11:162
249940,0watchdog/20-21swapper/220:11:192
249940,0watchdog/20-21swapper/219:56:192
179940,0migration/17606-21kthreadcore19:31:141
179940,0migration/17591-21fschecks_count19:56:101
179940,0migration/16978-21kthreadcore00:21:151
179940,0migration/16153-21kthreadcore21:01:181
179940,0migration/15056-21kthreadcore20:36:151
179940,0migration/13903-21kthreadcore00:01:171
179940,0migration/128387-21kthreadcore21:51:151
179940,0migration/123694-21sh21:31:121
179940,0migration/123382-21kthreadcore20:06:141
179940,0migration/112710-21mdstat19:11:161
179940,0migration/111934-21kthreadcore23:11:161
179940,0migration/110207-21irqrtprio21:41:131
179940,0migration/11013-21kthreadcore20:56:121
169940,0watchdog/19668-21ssh21:56:191
169940,0watchdog/16262-21timerwakeupswit23:06:241
169940,0watchdog/16236-21kthreadcore22:31:161
169940,0watchdog/13534-21sshd21:19:271
169940,0watchdog/128442-21munin-node19:26:151
169940,0watchdog/121680-21kthreadcore20:26:151
169940,0watchdog/114797-21perl00:26:121
169940,0watchdog/10-21swapper/119:46:191
149940,0migration/016993-21timerwakeupswit19:36:210
149940,0migration/01222-21kthreadcore22:11:170
65989932,0cyclictest4396-21awk19:31:122
65989932,0cyclictest16248-21kthreadcore20:46:132
65979932,0cyclictest7713-21fschecks_time23:46:131
65969933,0cyclictest5782-21kworker/0:119:15:570
65969933,0cyclictest24719-21kworker/0:222:26:100
65969933,0cyclictest24719-21kworker/0:222:01:020
65969933,0cyclictest24719-21kworker/0:221:06:000
65969933,0cyclictest24719-21kworker/0:220:40:590
65969932,0cyclictest8057-21awk23:46:150
339930,0migration/36763-21kthreadcore19:31:143
329930,0watchdog/30-21swapper/320:06:113
329930,0watchdog/30-21swapper/319:51:153
329930,0watchdog/30-21swapper/319:42:273
29805030,0irq/28-eth0-rx-0-21swapper/320:56:063
249930,0watchdog/226784-21kthreadcore20:31:152
249930,0watchdog/20-21swapper/219:10:592
179930,0migration/126919-21kthreadcore22:06:171
169930,0watchdog/10-21swapper/122:56:001
169930,0watchdog/10-21swapper/121:46:071
169930,0watchdog/10-21swapper/120:51:151
169930,0watchdog/10-21swapper/119:41:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional