You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-05 - 04:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Jan 05, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450710,0irq/25-eth10-21swapper/319:07:113
762323525,0sleep10-21swapper/119:06:431
755323526,0sleep20-21swapper/219:05:562
769223221,0sleep00-21swapper/019:07:300
77909976,0cyclictest16433-21cut23:08:130
77929960,0cyclictest0-21swapper/222:12:322
77909966,0cyclictest0-21swapper/000:28:090
339960,0migration/37988-21munin-node23:43:053
339960,0migration/37087-21sh00:00:523
339960,0migration/36240-21sh21:24:213
339960,0migration/329786-21sh21:18:473
29715060,0irq/26-eth0-rx-31910-21ssh00:15:283
29715060,0irq/26-eth0-rx-19202-21port_sshpriv23:08:173
29715060,0irq/26-eth0-rx-12867-21df_inode22:48:023
29715060,0irq/26-eth0-rx-10244-21iostat_ios00:23:073
249960,0watchdog/224449-21grep23:13:092
149960,0migration/024766-21sh21:37:370
149960,0migration/016714-21munin-node20:23:140
149960,0migration/011371-21sh23:03:200
149960,0migration/011371-21sh23:03:200
77929955,0cyclictest0-21swapper/223:49:012
77929955,0cyclictest0-21swapper/223:35:182
77929955,0cyclictest0-21swapper/223:18:072
77929955,0cyclictest0-21swapper/222:53:502
77929955,0cyclictest0-21swapper/222:33:212
77929955,0cyclictest0-21swapper/219:38:212
77929955,0cyclictest0-21swapper/200:10:092
77929950,0cyclictest0-21swapper/223:07:142
77929950,0cyclictest0-21swapper/223:07:142
77929950,0cyclictest0-21swapper/222:49:322
77929950,0cyclictest0-21swapper/222:27:502
77929950,0cyclictest0-21swapper/222:02:552
77929950,0cyclictest0-21swapper/221:57:432
77929950,0cyclictest0-21swapper/221:39:352
77919950,0cyclictest0-21swapper/100:26:191
77909955,0cyclictest6066-21diskmemload00:14:450
77909955,0cyclictest0-21swapper/023:14:030
77909955,0cyclictest0-21swapper/022:20:170
77909955,0cyclictest0-21swapper/022:04:120
77909955,0cyclictest0-21swapper/019:48:090
77909950,0cyclictest0-21swapper/023:55:120
77909950,0cyclictest0-21swapper/023:21:270
77909950,0cyclictest0-21swapper/000:25:470
339950,0migration/39864-21kthreadcore19:08:083
339950,0migration/331092-21kthreadcore21:58:093
339950,0migration/330305-21aten_r1power_po23:18:043
339950,0migration/329033-21latency_hist22:57:513
339950,0migration/328338-21irqrtprio22:18:083
339950,0migration/327674-21missed_timers19:43:083
339950,0migration/327017-21kthreadcore20:58:103
339950,0migration/325316-21aten_r1power_vo21:38:033
339950,0migration/323071-21kthreadcore22:53:063
339950,0migration/320689-21kthreadcore19:38:113
339950,0migration/319599-21idleruntime22:13:053
339950,0migration/315517-21unixbench-2d21:48:163
339950,0migration/314938-21fschecks_count23:28:033
339950,0migration/314626-21kthreadcore20:48:093
339950,0migration/31340-21kthreadcore21:03:133
339950,0migration/312461-21kthreadcore22:08:103
339950,0migration/31232-21fschecks_count00:18:053
29725050,0irq/27-eth0-tx-6332-21munin-node21:08:052
29725050,0irq/27-eth0-tx-27111-21sh21:58:072
29725050,0irq/27-eth0-tx-11050-21munin-node20:23:102
29725050,0irq/27-eth0-tx-10817-21kthreadcore21:48:092
29715050,0irq/26-eth0-rx-9772-21ssh22:24:273
29715050,0irq/26-eth0-rx-4350-21cpuspeed_turbos22:43:043
29715050,0irq/26-eth0-rx-32477-21kthreadcore23:38:093
29715050,0irq/26-eth0-rx-26058-21cpu00:33:053
29715050,0irq/26-eth0-rx-25292-21runrttasks21:56:393
29715050,0irq/26-eth0-rx-24193-21diskstats23:53:043
29715050,0irq/26-eth0-rx-17442-21munin-node00:08:053
29715050,0irq/26-eth0-rx-17257-21sendmail_mailtr22:28:133
29715050,0irq/26-eth0-rx-16826-21cut20:03:033
29715050,0irq/26-eth0-rx-10751-21kthreadcore21:28:083
29715050,0irq/26-eth0-rx-10204-21latency_hist22:07:513
29715050,0irq/26-eth0-rx-0-21swapper/323:48:083
29715050,0irq/26-eth0-rx-0-21swapper/323:36:283
29715050,0irq/26-eth0-rx-0-21swapper/323:13:093
29715050,0irq/26-eth0-rx-0-21swapper/323:03:363
29715050,0irq/26-eth0-rx-0-21swapper/323:03:363
29715050,0irq/26-eth0-rx-0-21swapper/322:37:553
29715050,0irq/26-eth0-rx-0-21swapper/322:33:343
29715050,0irq/26-eth0-rx-0-21swapper/321:45:403
29715050,0irq/26-eth0-rx-0-21swapper/321:33:103
29715050,0irq/26-eth0-rx-0-21swapper/321:13:093
29715050,0irq/26-eth0-rx-0-21swapper/300:28:163
29715050,0irq/26-eth0-rx-0-21swapper/300:02:593
27150,0ktimersoftd/20-21swapper/200:01:382
259950,0migration/26216-21latency00:18:102
259950,0migration/22829-21latency_hist22:22:512
259950,0migration/224385-21munin-node20:08:102
249950,0watchdog/222368-21processes21:33:132
179950,0migration/14805-21taskset23:02:451
179950,0migration/14112-21kthreadcore21:23:091
179950,0migration/130600-21fschecks_time20:13:081
179950,0migration/129594-21kthreadcore22:18:081
179950,0migration/126906-21kthreadcore00:13:091
179950,0migration/126157-21taskset22:16:191
179950,0migration/125680-21sh21:57:361
179950,0migration/122359-21sh21:16:301
179950,0migration/121657-21taskset23:48:221
179950,0migration/119644-21kthreadcore21:33:091
179950,0migration/118956-21kthreadcore00:08:071
179950,0migration/113070-21timerwakeupswit23:43:141
149950,0migration/09211-21kthreadcore23:23:100
149950,0migration/08485-21fschecks_count19:08:040
149950,0migration/08092-21kthreadcore22:23:100
149950,0migration/06982-21fschecks_time21:08:050
149950,0migration/06728-21kthreadcore20:43:070
149950,0migration/03337-21sh22:59:150
149950,0migration/032551-21sh21:59:000
149950,0migration/031793-21kthreadcore20:13:120
149950,0migration/028880-21latency_hist19:47:510
149950,0migration/028700-21taskset22:57:310
149950,0migration/026737-21kthreadcore23:33:090
149950,0migration/02491-21kthreadcore20:38:100
149950,0migration/021294-21processes19:38:140
149950,0migration/020430-21munin-node21:13:130
149950,0migration/019716-21kthreadcore21:53:090
149950,0migration/019605-21sh22:50:350
149950,0migration/010599-21kthreadcore23:43:080
149950,0migration/010133-21kthreadcore00:03:080
77929943,0cyclictest3556-21lldpd20:51:052
77929943,0cyclictest3556-21lldpd20:00:352
77929943,0cyclictest3514-21snmpd20:16:472
77929943,0cyclictest3241-21irqbalance23:37:592
77929943,0cyclictest3241-21irqbalance19:18:002
77929943,0cyclictest29872-21awk19:48:122
77929943,0cyclictest25206-21awk20:33:062
77929943,0cyclictest24243-21awk21:18:082
77929943,0cyclictest15657-21awk21:13:072
77919944,0cyclictest0-21swapper/119:18:051
77919943,0cyclictest9152-21awk00:03:061
77919943,0cyclictest8855-21awk19:08:061
77919943,0cyclictest545-21awk23:58:081
77919943,0cyclictest3241-21irqbalance22:38:091
77919943,0cyclictest31096-21awk23:18:071
77919943,0cyclictest2862-21sort22:03:051
77919943,0cyclictest26042-21awk21:38:061
77919943,0cyclictest25333-21awk20:33:071
77919943,0cyclictest22722-21awk23:13:061
77919943,0cyclictest18424-21awk00:28:071
77919943,0cyclictest15681-21awk19:13:101
77919943,0cyclictest14531-21awk23:08:111
77919943,0cyclictest13387-21awk22:48:051
77919943,0cyclictest12341-21awk22:28:061
77919940,0cyclictest0-21swapper/121:48:201
77909944,0cyclictest0-21swapper/023:32:490
77909944,0cyclictest0-21swapper/022:08:370
77909944,0cyclictest0-21swapper/021:48:070
77909943,0cyclictest367-21awk21:23:060
77909943,0cyclictest3514-21snmpd19:30:170
77909943,0cyclictest3241-21irqbalance21:29:000
77909943,0cyclictest26082-21awk21:38:060
77909943,0cyclictest24249-21awk20:08:090
77909943,0cyclictest20732-21awk22:33:070
77909943,0cyclictest2008-21sadc19:27:510
77909943,0cyclictest19692-21awk22:13:060
77909943,0cyclictest1904-21awk21:43:050
77909943,0cyclictest18586-21awk20:28:070
77909940,0cyclictest16508-21basename20:02:510
4140,0ktimersoftd/00-21swapper/000:34:120
339940,0migration/39598-21fschecks_time19:33:063
339940,0migration/39239-21timerwakeupswit19:53:163
339940,0migration/330528-21kthreadcore19:23:123
339940,0migration/326135-21kthreadcore19:18:123
339940,0migration/324763-21runrttasks20:53:593
339940,0migration/324501-21irqcore20:08:103
339940,0migration/31983-21kthreadcore20:13:133
29725040,0irq/27-eth0-tx-22329-21munin-node19:43:082
29725040,0irq/27-eth0-tx-17704-21munin-node20:28:062
29725040,0irq/27-eth0-tx-0-21swapper/223:52:552
29725040,0irq/27-eth0-tx-0-21swapper/223:43:082
29725040,0irq/27-eth0-tx-0-21swapper/223:28:522
29725040,0irq/27-eth0-tx-0-21swapper/222:58:112
29725040,0irq/27-eth0-tx-0-21swapper/222:38:112
29725040,0irq/27-eth0-tx-0-21swapper/222:30:032
29725040,0irq/27-eth0-tx-0-21swapper/222:13:022
29725040,0irq/27-eth0-tx-0-21swapper/221:42:582
29725040,0irq/27-eth0-tx-0-21swapper/221:27:552
29725040,0irq/27-eth0-tx-0-21swapper/219:08:042
29725040,0irq/27-eth0-tx-0-21swapper/200:33:182
29725040,0irq/27-eth0-tx-0-21swapper/200:24:302
29725040,0irq/27-eth0-tx-0-21swapper/200:03:072
29715040,0irq/26-eth0-rx-0-21swapper/323:23:063
29715040,0irq/26-eth0-rx-0-21swapper/321:08:093
29715040,0irq/26-eth0-rx-0-21swapper/320:18:053
259940,0migration/228165-21aten2.4-expect19:23:072
259940,0migration/218724-21aten_r1power_cu20:53:032
259940,0migration/214925-21cpuspeed_turbos19:13:062
249940,0watchdog/26479-21df23:23:052
249940,0watchdog/24727-21grep22:43:052
249940,0watchdog/231911-21ssh21:23:012
249940,0watchdog/227376-21cat00:13:092
249940,0watchdog/219807-21kthreadcore20:03:102
249940,0watchdog/214162-21tune2fs23:08:092
249940,0watchdog/212541-21kthreadcore19:33:102
249940,0watchdog/20-21swapper/220:38:102
249940,0watchdog/20-21swapper/219:53:102
249940,0watchdog/20-21swapper/200:28:052
179940,0migration/19582-21kthreadcore21:08:091
179940,0migration/17631-21kthreadcore22:23:101
179940,0migration/13634-21munin-node19:53:051
179940,0migration/129550-21fschecks_time19:48:101
179940,0migration/129362-21kthreadcore21:58:091
179940,0migration/126593-21sh22:53:111
179940,0migration/125786-21latency_hist00:32:501
179940,0migration/124825-21kthreadcore21:18:091
179940,0migration/122777-21fschecks_count19:43:031
179940,0migration/118862-21aten2.4-expect20:53:031
179940,0migration/116130-21kthreadcore20:48:091
179940,0migration/115695-21aten_r1power_cu19:38:041
179940,0migration/114820-21kthreadcore22:08:111
179940,0migration/110691-21kthreadcore19:33:091
169940,0watchdog/15606-21irqstats20:18:101
169940,0watchdog/132542-21fschecks_count21:03:091
169940,0watchdog/127677-21tr23:53:091
169940,0watchdog/12754-21kthreadcore00:18:091
169940,0watchdog/120593-21grep20:28:101
169940,0watchdog/120317-21fschecks_count22:33:051
169940,0watchdog/115716-21ssh21:31:291
169940,0watchdog/114479-21ssh23:27:571
169940,0watchdog/11066-21kthreadcore20:38:101
169940,0watchdog/10-21swapper/123:03:331
169940,0watchdog/10-21swapper/123:03:331
169940,0watchdog/10-21swapper/122:43:051
169940,0watchdog/10-21swapper/120:03:101
169940,0watchdog/10-21swapper/119:58:101
149940,0migration/09072-21kthreadcore20:18:110
149940,0migration/08321-21sh00:21:220
149940,0migration/06508-21runrttasks23:42:050
149940,0migration/028232-21kthreadcore21:18:110
149940,0migration/026875-21kthreadcore20:58:100
149940,0migration/024093-21kthreadcore19:18:110
149940,0migration/022986-21runrttasks00:09:010
149940,0migration/022175-21ntp_states20:03:120
149940,0migration/019039-21kthreadcore23:48:080
149940,0migration/010560-21sh22:43:550
77929933,0cyclictest3556-21lldpd20:18:052
77929932,0cyclictest26317-21awk20:58:082
77929932,0cyclictest10577-21expr20:43:082
77919932,0cyclictest3556-21lldpd21:47:361
77919932,0cyclictest28859-21awk19:23:091
77919932,0cyclictest24600-21irqrtprio20:08:101
77919932,0cyclictest11932-21awk20:23:061
77919930,0cyclictest30670-21grep20:58:121
77909932,0cyclictest9801-21sadc19:57:510
77909932,0cyclictest604-21awk23:58:080
77909932,0cyclictest3514-21snmpd19:33:020
77909932,0cyclictest3241-21irqbalance22:28:390
77909932,0cyclictest3241-21irqbalance20:48:100
77909930,0cyclictest23516-21tr20:53:100
339930,0migration/324097-21runrttasks20:32:373
29715030,0irq/26-eth0-rx-0-21swapper/320:33:163
29715030,0irq/26-eth0-rx-0-21swapper/320:23:023
29715030,0irq/26-eth0-rx-0-21swapper/319:13:053
249930,0watchdog/20-21swapper/221:03:092
249930,0watchdog/20-21swapper/219:28:062
169930,0watchdog/10-21swapper/123:38:091
169930,0watchdog/10-21swapper/123:33:011
169930,0watchdog/10-21swapper/123:23:091
169930,0watchdog/10-21swapper/119:28:061
77919921,0cyclictest0-21swapper/120:42:511
77909920,0cyclictest6066-21diskmemload22:37:520
77909920,0cyclictest14635-21latency_hist19:12:510
77909920,0cyclictest0-21swapper/021:02:510
77909920,0cyclictest0-21swapper/020:32:510
329920,0watchdog/30-21swapper/319:48:153
29715020,0irq/26-eth0-rx-0-21swapper/320:43:043
29715020,0irq/26-eth0-rx-0-21swapper/320:38:123
29715020,0irq/26-eth0-rx-0-21swapper/319:57:553
29715020,0irq/26-eth0-rx-0-21swapper/319:28:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional