You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 01:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Dec 25, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450710,0irq/25-eth10-21swapper/319:05:293
247623829,0sleep20-21swapper/219:06:102
247923623,0sleep00-21swapper/019:06:120
251623424,0sleep10-21swapper/119:06:371
35160,0ktimersoftd/30-21swapper/321:24:403
339960,0migration/329783-21sh22:19:423
339960,0migration/327138-21sh23:18:503
339960,0migration/316462-21sh22:51:413
339960,0migration/315070-21kthreadcore21:14:113
339960,0migration/313596-21kthreadcore21:34:103
29715060,0irq/26-eth0-rx-6754-21ssh22:09:033
29715060,0irq/26-eth0-rx-526-21ssh23:00:563
29715060,0irq/26-eth0-rx-27781-21df23:19:043
29715060,0irq/26-eth0-rx-23085-21munin-node22:54:113
29715060,0irq/26-eth0-rx-13944-21ssh00:07:503
29715060,0irq/26-eth0-rx-10937-21processes21:49:133
27949960,0cyclictest0-21swapper/123:11:421
179960,0migration/13193-21munin-node23:04:081
149960,0migration/04885-21sh00:01:150
149960,0migration/03001-21sh23:23:240
149960,0migration/028223-21sh23:38:140
149960,0migration/017959-21sh21:18:260
149960,0migration/011208-21sh23:44:390
339950,0migration/38939-21kthreadcore19:34:113
339950,0migration/38743-21kthreadcore20:49:123
339950,0migration/329114-21kthreadcore20:39:083
339950,0migration/325788-21kthreadcore00:14:093
339950,0migration/325223-21fschecks_count22:39:063
339950,0migration/324889-21kthreadcore23:34:103
339950,0migration/32458-21kthreadcore22:44:093
339950,0migration/322836-21sh00:30:383
339950,0migration/322098-21kthreadcore21:39:103
339950,0migration/321110-21kthreadcore21:19:093
339950,0migration/315546-21kthreadcore23:29:113
339950,0migration/315542-21kthreadcore21:54:093
339950,0migration/312985-21cpuspeed_turbos20:29:063
29725050,0irq/27-eth0-tx-9821-21ssh21:29:272
29725050,0irq/27-eth0-tx-6725-21kthreadcore23:24:122
29725050,0irq/27-eth0-tx-5454-21ssh22:06:552
29725050,0irq/27-eth0-tx-31464-21ssh22:23:412
29725050,0irq/27-eth0-tx-25037-21munin-node20:14:152
29725050,0irq/27-eth0-tx-17666-21kthreadcore22:34:092
29725050,0irq/27-eth0-tx-17393-21munin-node19:44:122
29725050,0irq/27-eth0-tx-16397-21munin-node19:19:072
29725050,0irq/27-eth0-tx-0-21swapper/223:51:362
29725050,0irq/27-eth0-tx-0-21swapper/223:41:262
29725050,0irq/27-eth0-tx-0-21swapper/223:07:272
29725050,0irq/27-eth0-tx-0-21swapper/222:53:562
29725050,0irq/27-eth0-tx-0-21swapper/222:40:222
29725050,0irq/27-eth0-tx-0-21swapper/222:09:582
29725050,0irq/27-eth0-tx-0-21swapper/222:01:122
29725050,0irq/27-eth0-tx-0-21swapper/221:39:092
29715050,0irq/26-eth0-rx-5196-21sed00:19:133
29715050,0irq/26-eth0-rx-3396-21sh21:28:533
29715050,0irq/26-eth0-rx-30794-21ssh00:34:363
29715050,0irq/26-eth0-rx-2838-21perl23:04:063
29715050,0irq/26-eth0-rx-21901-21kthreadcore19:44:113
29715050,0irq/26-eth0-rx-18687-21kthreadcore22:34:103
29715050,0irq/26-eth0-rx-17828-21kthreadcore00:09:103
29715050,0irq/26-eth0-rx-1728-21grep20:44:103
29715050,0irq/26-eth0-rx-1728-21grep20:44:103
29715050,0irq/26-eth0-rx-15433-21ssh22:32:273
29715050,0irq/26-eth0-rx-15333-21taskset00:27:423
29715050,0irq/26-eth0-rx-13412-21aten_r1power_fr23:49:043
29715050,0irq/26-eth0-rx-0-21swapper/323:59:143
29715050,0irq/26-eth0-rx-0-21swapper/323:54:043
29715050,0irq/26-eth0-rx-0-21swapper/323:39:003
29715050,0irq/26-eth0-rx-0-21swapper/323:24:013
29715050,0irq/26-eth0-rx-0-21swapper/323:12:523
29715050,0irq/26-eth0-rx-0-21swapper/322:24:283
29715050,0irq/26-eth0-rx-0-21swapper/322:14:103
29715050,0irq/26-eth0-rx-0-21swapper/322:00:353
29715050,0irq/26-eth0-rx-0-21swapper/321:46:023
29715050,0irq/26-eth0-rx-0-21swapper/321:09:453
29715050,0irq/26-eth0-rx-0-21swapper/320:14:103
29715050,0irq/26-eth0-rx-0-21swapper/319:39:113
27959955,0cyclictest20577-21ssh22:14:112
27959955,0cyclictest0-21swapper/222:24:072
27959950,0cyclictest0-21swapper/200:08:582
27949955,0cyclictest0-21swapper/122:56:251
27949955,0cyclictest0-21swapper/122:38:451
27949955,0cyclictest0-21swapper/121:39:571
27949955,0cyclictest0-21swapper/121:23:141
27949955,0cyclictest0-21swapper/120:54:041
27949950,0cyclictest0-21swapper/123:30:091
27949950,0cyclictest0-21swapper/121:46:441
27939955,0cyclictest0-21swapper/023:57:110
27939955,0cyclictest0-21swapper/023:15:520
27939955,0cyclictest0-21swapper/022:54:160
27939955,0cyclictest0-21swapper/022:00:310
27939953,0cyclictest32464-21kworker/0:022:59:080
27939953,0cyclictest32464-21kworker/0:022:26:520
27939953,0cyclictest32464-21kworker/0:022:08:540
27939953,0cyclictest32464-21kworker/0:021:44:220
27939953,0cyclictest32464-21kworker/0:020:59:520
27939953,0cyclictest32464-21kworker/0:020:52:520
27939953,0cyclictest32464-21kworker/0:020:39:250
27939953,0cyclictest32464-21kworker/0:020:37:230
27939953,0cyclictest32464-21kworker/0:020:24:550
27939953,0cyclictest32464-21kworker/0:020:22:230
27939953,0cyclictest27275-21kworker/0:219:54:110
27939953,0cyclictest27275-21kworker/0:219:46:550
27939953,0cyclictest27275-21kworker/0:219:39:250
27939953,0cyclictest27275-21kworker/0:219:37:530
27939953,0cyclictest27275-21kworker/0:219:33:230
27939953,0cyclictest27275-21kworker/0:219:21:550
27939953,0cyclictest27275-21kworker/0:219:16:230
27939952,0cyclictest32464-21kworker/0:020:29:090
27939950,0cyclictest0-21swapper/022:29:030
27939950,0cyclictest0-21swapper/021:23:470
259950,0migration/29805-21kthreadcore00:04:102
259950,0migration/27172-21sh22:44:352
259950,0migration/25196-21taskset19:59:072
259950,0migration/232643-21if_eth1.220:19:082
259950,0migration/230123-21df_abs19:29:042
259950,0migration/223535-21fschecks_count00:14:052
259950,0migration/223246-21sh00:31:372
259950,0migration/221659-21sh21:58:302
259950,0migration/215822-21kthreadcore23:29:112
19150,0ktimersoftd/10-21swapper/100:21:521
179950,0migration/14668-21kthreadcore22:24:091
179950,0migration/132311-21latency21:04:111
179950,0migration/127095-21munin-node22:59:081
179950,0migration/125999-21munin-node23:34:131
179950,0migration/124170-21timerandwakeup20:09:161
169950,0watchdog/16499-21kthreadcore21:09:111
169950,0watchdog/14166-21tune2fs21:29:071
169950,0watchdog/117346-21grep20:29:111
169950,0watchdog/112956-21irqrtprio20:04:111
169950,0watchdog/10-21swapper/120:24:111
169950,0watchdog/10-21swapper/100:29:101
149950,0migration/05005-21kthreadcore19:09:090
149950,0migration/04274-21kthreadcore21:09:110
149950,0migration/032707-21kthreadcore23:39:100
149950,0migration/030598-21aten_r1power_en22:04:030
149950,0migration/030399-21kthreadcore21:24:090
149950,0migration/02813-21fschecks_time23:04:060
149950,0migration/024526-21latency_hist22:38:520
149950,0migration/024006-21kthreadcore21:39:110
149950,0migration/023079-21munin-node19:24:030
149950,0migration/022988-21latency_hist00:13:520
149950,0migration/022522-21kthreadcore20:09:110
149950,0migration/021901-21sh00:29:150
149950,0migration/019004-21taskset23:31:210
149950,0migration/016406-21kthreadcore23:49:100
149950,0migration/016027-21munin-node00:09:090
149950,0migration/015571-21kthreadcore21:34:100
149950,0migration/013581-21latency_hist21:53:520
149950,0migration/013502-21smart_sda00:24:140
149950,0migration/013168-21taskset00:06:000
149950,0migration/012795-21irqcore20:04:110
149950,0migration/010931-21sh21:31:510
149950,0migration/010804-21switchtime19:59:180
339940,0migration/326967-21kthreadcore19:24:093
339940,0migration/323798-21kthreadcore20:59:103
339940,0migration/314852-21irqcore20:54:093
29725040,0irq/27-eth0-tx-3524-21sshd20:26:162
29725040,0irq/27-eth0-tx-3009-21munin-node19:09:052
29725040,0irq/27-eth0-tx-29201-21ssh21:44:022
29725040,0irq/27-eth0-tx-18460-21ssh21:18:592
29725040,0irq/27-eth0-tx-0-21swapper/223:59:372
29725040,0irq/27-eth0-tx-0-21swapper/223:53:542
29725040,0irq/27-eth0-tx-0-21swapper/223:44:232
29725040,0irq/27-eth0-tx-0-21swapper/223:34:042
29725040,0irq/27-eth0-tx-0-21swapper/223:18:572
29725040,0irq/27-eth0-tx-0-21swapper/223:09:112
29725040,0irq/27-eth0-tx-0-21swapper/222:58:542
29725040,0irq/27-eth0-tx-0-21swapper/222:49:202
29725040,0irq/27-eth0-tx-0-21swapper/222:28:592
29725040,0irq/27-eth0-tx-0-21swapper/221:34:432
29725040,0irq/27-eth0-tx-0-21swapper/221:14:522
29725040,0irq/27-eth0-tx-0-21swapper/200:35:252
29725040,0irq/27-eth0-tx-0-21swapper/200:24:062
29725040,0irq/27-eth0-tx-0-21swapper/200:19:222
29715040,0irq/26-eth0-rx-0-21swapper/323:44:273
29715040,0irq/26-eth0-rx-0-21swapper/322:04:563
29715040,0irq/26-eth0-rx-0-21swapper/320:34:113
27959943,0cyclictest24929-21awk19:49:082
27959943,0cyclictest23975-21awk19:24:072
27959943,0cyclictest12588-21awk20:04:102
27959940,0cyclictest0-21swapper/221:10:012
27949943,0cyclictest7286-21awk00:04:081
27949943,0cyclictest5858-21awk23:44:071
27949943,0cyclictest4620-21tune2fs19:34:061
27949943,0cyclictest3556-21lldpd22:44:231
27949943,0cyclictest3556-21lldpd22:41:531
27949943,0cyclictest3556-21lldpd22:19:241
27949943,0cyclictest3556-21lldpd22:02:241
27949943,0cyclictest3556-21lldpd21:54:231
27949943,0cyclictest3556-21lldpd21:34:531
27949943,0cyclictest3556-21lldpd20:58:541
27949943,0cyclictest3556-21lldpd19:54:231
27949943,0cyclictest3556-21lldpd19:14:531
27949943,0cyclictest3514-21snmpd20:17:211
27949943,0cyclictest3241-21irqbalance20:39:521
27949943,0cyclictest30625-21awk19:29:071
27949943,0cyclictest29671-21awk23:39:081
27949943,0cyclictest19905-21awk23:14:081
27949943,0cyclictest18261-21awk19:44:081
27949943,0cyclictest10804-21awk21:14:091
27939943,0cyclictest23891-21kworker/0:200:20:120
27939942,0cyclictest32464-21kworker/0:020:48:240
27939942,0cyclictest32464-21kworker/0:020:48:240
259940,0migration/29132-21kthreadcore21:49:102
259940,0migration/226606-21fschecks_time20:39:052
259940,0migration/221803-21kthreadcore23:14:102
259940,0migration/221761-21kthreadcore20:09:112
259940,0migration/219496-21aten_r1power_fr20:34:052
259940,0migration/212614-21proc_pri20:49:152
259940,0migration/211738-21munin-node19:39:102
249940,0watchdog/2648-21timerwakeupswit21:24:152
249940,0watchdog/25361-21kthreadcore20:44:112
249940,0watchdog/25361-21kthreadcore20:44:112
249940,0watchdog/232723-21python21:04:152
249940,0watchdog/216071-21runrttasks19:17:562
249940,0watchdog/20-21swapper/220:29:112
179940,0migration/19442-21kthreadcore22:29:121
179940,0migration/17771-21aten_r1power_fr00:24:031
179940,0migration/17280-21cpu20:49:071
179940,0migration/17123-21fschecks_count22:09:051
179940,0migration/15049-21sensors_temp20:19:141
179940,0migration/15045-21kthreadcore19:09:091
179940,0migration/14136-21sh23:59:331
179940,0migration/13826-21cpuspeed_turbos23:24:071
179940,0migration/132730-21ntp_kernel_pll_21:24:121
179940,0migration/13268-21kthreadcore20:44:111
179940,0migration/13268-21kthreadcore20:44:111
179940,0migration/130346-21taskset00:16:511
179940,0migration/124998-21munin-node00:34:041
179940,0migration/121718-21kthreadcore20:34:101
179940,0migration/117994-21kthreadcore00:09:101
179940,0migration/116836-21fschecks_count19:19:071
179940,0migration/11082-21unixbench-2d23:19:151
169940,0watchdog/16559-21runrttasks23:55:351
169940,0watchdog/14949-21ssh22:05:431
169940,0watchdog/129469-21munin-node19:49:121
169940,0watchdog/120164-21ssh23:50:511
169940,0watchdog/112882-21sh21:52:431
169940,0watchdog/1111702sleep120:00:071
169940,0watchdog/11070-21diskmemload22:49:071
169940,0watchdog/10-21swapper/119:39:081
149940,0migration/06588-21kthreadcore21:49:090
149940,0migration/028289-21sh22:19:100
149940,0migration/016538-21kthreadcore20:54:100
339930,0migration/34851-21munin-node20:19:133
29725030,0irq/27-eth0-tx-0-21swapper/219:34:052
29715030,0irq/26-eth0-rx-0-21swapper/321:04:053
29715030,0irq/26-eth0-rx-0-21swapper/320:24:063
29715030,0irq/26-eth0-rx-0-21swapper/320:09:053
29715030,0irq/26-eth0-rx-0-21swapper/319:59:073
29715030,0irq/26-eth0-rx-0-21swapper/319:54:063
29715030,0irq/26-eth0-rx-0-21swapper/319:49:543
29715030,0irq/26-eth0-rx-0-21swapper/319:29:513
29715030,0irq/26-eth0-rx-0-21swapper/319:19:103
29715030,0irq/26-eth0-rx-0-21swapper/319:14:113
29715030,0irq/26-eth0-rx-0-21swapper/319:09:053
27959932,0cyclictest20364-21munin-node20:59:082
27959932,0cyclictest13713-21munin-node20:54:052
27959930,0cyclictest30702-21munin-node19:54:052
27949932,0cyclictest23936-21awk19:24:061
27939933,0cyclictest32464-21kworker/0:023:24:360
27939933,0cyclictest32464-21kworker/0:023:09:040
27939933,0cyclictest32464-21kworker/0:022:39:040
27939933,0cyclictest32464-21kworker/0:022:14:040
27939933,0cyclictest32464-21kworker/0:021:04:000
27939933,0cyclictest27275-21kworker/0:220:14:030
27939933,0cyclictest27275-21kworker/0:219:48:570
27939932,0cyclictest25431-21awk00:34:050
27939932,0cyclictest1821-21grep22:44:080
27939932,0cyclictest10242-21awk22:49:090
169930,0watchdog/10-21swapper/122:14:111
35120,0ktimersoftd/312558-21taskset20:04:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional