You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 12:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Jul 12, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
626423524,0sleep00-21swapper/018:49:390
597823524,0sleep20-21swapper/218:46:252
3268623523,0sleep10-21swapper/118:45:201
304450350,0irq/25-eth10-21swapper/318:45:513
63809960,0cyclictest0-21swapper/021:34:350
339960,0migration/322427-21latency20:35:243
339960,0migration/320588-21sh21:18:403
339960,0migration/320525-21sh23:40:063
339960,0migration/319406-21sh23:57:393
29725060,0irq/27-eth0-tx-4746-21cat20:50:040
29715060,0irq/26-eth0-rx-30381-21tr19:05:243
29715060,0irq/26-eth0-rx-24247-21diskstats20:40:183
29715060,0irq/26-eth0-rx-21768-21sed21:20:163
149960,0migration/020709-21sh22:19:590
149960,0migration/019713-21sh22:57:520
149960,0migration/015661-21munin-node19:45:170
8215050,0irq/24-0000:00:3615-21grep20:25:052
63829955,0cyclictest0-21swapper/221:33:372
63829950,0cyclictest0-21swapper/223:25:332
63829950,0cyclictest0-21swapper/221:03:202
63829950,0cyclictest0-21swapper/219:47:592
63819955,0cyclictest0-21swapper/122:56:161
63819955,0cyclictest0-21swapper/122:25:031
63819950,0cyclictest0-21swapper/123:19:081
63819950,0cyclictest0-21swapper/121:15:221
63819950,0cyclictest0-21swapper/120:50:271
63809954,0cyclictest0-21swapper/022:40:190
63809950,0cyclictest0-21swapper/020:45:230
339950,0migration/38834-21kthreadcore20:25:223
339950,0migration/37562-21kthreadcore23:10:203
339950,0migration/35196-21kthreadcore19:10:243
339950,0migration/34603-21cpuspeed_turbos00:10:163
339950,0migration/34149-21runrttasks21:28:073
339950,0migration/33642-21runrttasks20:00:193
339950,0migration/331775-21kthreadcore23:45:213
339950,0migration/329071-21latency_hist22:05:043
339950,0migration/327697-21kthreadcore20:15:223
339950,0migration/32760-21switchtime21:05:263
339950,0migration/327508-21sh22:01:123
339950,0migration/322757-21idleruntime19:50:173
339950,0migration/32215-21kthreadcore20:45:243
339950,0migration/321270-21gltestperf00:00:183
339950,0migration/321126-21latency_hist21:40:043
339950,0migration/320574-21taskset19:23:573
339950,0migration/320486-21taskset20:59:083
339950,0migration/320322-21sh23:19:593
339950,0migration/318900-21taskset22:55:313
339950,0migration/318167-21sh00:15:283
339950,0migration/315577-21taskset20:05:443
339950,0migration/314564-21kthreadcore22:15:213
339950,0migration/31181-21taskset19:30:533
339950,0migration/310527-21munin-node21:30:223
29725050,0irq/27-eth0-tx-6828-21cut22:10:212
29725050,0irq/27-eth0-tx-4411-21munin-node23:50:172
29725050,0irq/27-eth0-tx-32528-21kthreadcore22:05:222
29725050,0irq/27-eth0-tx-28177-21ssh22:43:322
29725050,0irq/27-eth0-tx-25501-21kthreadcore23:00:212
29725050,0irq/27-eth0-tx-21985-21perl21:20:172
29725050,0irq/27-eth0-tx-12108-21ssh21:53:112
29725050,0irq/27-eth0-tx-0-21swapper/223:58:302
29725050,0irq/27-eth0-tx-0-21swapper/222:52:322
29725050,0irq/27-eth0-tx-0-21swapper/221:42:162
29725050,0irq/27-eth0-tx-0-21swapper/200:05:212
29725050,0irq/27-eth0-tx-0-21swapper/020:35:240
29725050,0irq/27-eth0-tx-0-21swapper/019:51:210
29725050,0irq/27-eth0-tx-0-21swapper/019:25:190
29725050,0irq/27-eth0-tx-0-21swapper/019:24:210
29725050,0irq/27-eth0-tx-0-21swapper/019:00:260
29715050,0irq/26-eth0-rx-4811-21ssh20:50:043
29715050,0irq/26-eth0-rx-3084-21ssh22:26:203
29715050,0irq/26-eth0-rx-29064-21tail22:45:163
29715050,0irq/26-eth0-rx-23395-21cat23:00:203
29715050,0irq/26-eth0-rx-21152-21munin-node19:45:283
29715050,0irq/26-eth0-rx-19869-21ssh23:38:533
29715050,0irq/26-eth0-rx-17638-21cut21:35:253
29715050,0irq/26-eth0-rx-15673-21kthreadcore22:35:223
29715050,0irq/26-eth0-rx-13539-21cut21:55:163
29715050,0irq/26-eth0-rx-12256-21chrt21:12:523
29715050,0irq/26-eth0-rx-11264-21sh21:50:483
29715050,0irq/26-eth0-rx-0-21swapper/323:50:213
29715050,0irq/26-eth0-rx-0-21swapper/323:32:153
29715050,0irq/26-eth0-rx-0-21swapper/323:29:163
29715050,0irq/26-eth0-rx-0-21swapper/323:05:223
29715050,0irq/26-eth0-rx-0-21swapper/322:40:473
29715050,0irq/26-eth0-rx-0-21swapper/322:30:073
29715050,0irq/26-eth0-rx-0-21swapper/322:13:103
29715050,0irq/26-eth0-rx-0-21swapper/321:03:373
29715050,0irq/26-eth0-rx-0-21swapper/300:09:343
27150,0ktimersoftd/20-21swapper/222:26:352
27150,0ktimersoftd/20-21swapper/222:19:392
259950,0migration/24730-21latency_hist20:50:042
259950,0migration/228749-21latency_hist23:05:042
259950,0migration/227012-21timerwakeupswit22:20:272
259950,0migration/220084-21timerwakeupswit19:20:272
259950,0migration/217998-21idleruntime20:35:202
259950,0migration/215002-21kthreadcore22:55:222
259950,0migration/210143-21kthreadcore21:10:222
19150,0ktimersoftd/10-21swapper/123:40:441
179950,0migration/15962-21kthreadcore23:50:201
179950,0migration/125178-21latency19:00:241
179950,0migration/123898-21kthreadcore19:25:211
179950,0migration/120799-21sh21:39:441
179950,0migration/112344-21taskset23:15:041
169950,0watchdog/10-21swapper/123:35:221
149950,0migration/09117-21kthreadcore18:50:220
149950,0migration/08688-21kthreadcore23:30:220
149950,0migration/06534-21kthreadcore23:10:200
149950,0migration/06155-21kthreadcore22:50:200
149950,0migration/0600-21latency_hist19:10:050
149950,0migration/03075-21sh22:26:190
149950,0migration/029729-21gltestperf22:45:180
149950,0migration/028162-21sh21:22:150
149950,0migration/028100-21latency20:15:220
149950,0migration/02572-21proc_pri21:05:240
149950,0migration/02416-21mdstat21:25:230
149950,0migration/023737-21kthreadcore22:20:210
149950,0migration/023433-21kthreadcore22:00:210
149950,0migration/022445-21kthreadcore23:00:200
149950,0migration/021897-21hwlatdetect21:00:190
149950,0migration/021373-21aten_r1power_vo21:40:180
149950,0migration/020639-21munin-node23:40:150
149950,0migration/019725-21taskset21:16:110
149950,0migration/019164-21sh23:16:440
149950,0migration/014820-21kthreadcore23:55:200
149950,0migration/013637-21kthreadcore20:30:220
149950,0migration/013497-21cpuspeed_turbos18:55:180
149950,0migration/012074-21taskset21:12:270
149950,0migration/011378-21sh22:11:220
149950,0migration/01126-21kthreadcore00:05:220
149950,0migration/010911-21kthreadcore20:05:230
149950,0migration/010905-21munin-node22:30:290
63829944,0cyclictest0-21swapper/219:50:112
63829943,0cyclictest3514-21snmpd21:05:172
63829943,0cyclictest3514-21snmpd20:40:472
63829943,0cyclictest3514-21snmpd20:30:472
63829943,0cyclictest3514-21snmpd20:17:472
63829943,0cyclictest3514-21snmpd20:04:172
63829943,0cyclictest3514-21snmpd19:57:322
63829943,0cyclictest3514-21snmpd19:40:162
63829943,0cyclictest3514-21snmpd19:36:462
63829943,0cyclictest3514-21snmpd19:30:462
63829943,0cyclictest3514-21snmpd19:15:462
63829943,0cyclictest3514-21snmpd19:10:462
63829943,0cyclictest3514-21snmpd19:00:462
63829943,0cyclictest3514-21snmpd18:52:462
63829943,0cyclictest30618-21basename20:45:182
63829943,0cyclictest13348-21awk00:15:212
63829943,0cyclictest10433-21awk20:05:212
63819944,0cyclictest0-21swapper/122:26:471
63819944,0cyclictest0-21swapper/121:40:361
63819944,0cyclictest0-21swapper/100:05:191
63819943,0cyclictest7593-21awk18:50:211
63819943,0cyclictest3556-21lldpd23:58:521
63819943,0cyclictest3556-21lldpd21:55:511
63819943,0cyclictest3556-21lldpd19:58:511
63819943,0cyclictest3556-21lldpd19:42:501
63819943,0cyclictest3556-21lldpd19:22:511
63819943,0cyclictest3556-21lldpd00:12:511
63819943,0cyclictest3556-21lldpd00:00:521
63819943,0cyclictest3514-21snmpd21:45:331
63819943,0cyclictest30137-21awk21:05:171
63819943,0cyclictest21958-21awk21:00:191
63819943,0cyclictest21533-21awk23:20:181
63819943,0cyclictest13904-21awk22:15:191
63809944,0cyclictest0-21swapper/023:22:530
63809944,0cyclictest0-21swapper/000:03:340
63809943,0cyclictest5232-21awk00:10:190
63809943,0cyclictest4694-21sadc21:50:040
63809943,0cyclictest3514-21snmpd23:50:200
63809943,0cyclictest3514-21snmpd23:35:050
63809943,0cyclictest3241-21irqbalance23:29:120
63809943,0cyclictest3241-21irqbalance22:07:130
63809943,0cyclictest3241-21irqbalance21:59:130
63809943,0cyclictest3241-21irqbalance19:40:230
63809943,0cyclictest3241-21irqbalance19:10:230
63809943,0cyclictest3241-21irqbalance00:16:320
63809943,0cyclictest20637-21sadc22:40:040
63809943,0cyclictest13713-21kworker/0:221:50:140
339940,0migration/331098-21kthreadcore21:45:213
339940,0migration/317039-21kthreadcore18:55:233
339940,0migration/310852-21kthreadcore19:15:223
29725040,0irq/27-eth0-tx-7280-21grep00:10:212
29725040,0irq/27-eth0-tx-13039-21aten2.4-expect20:55:192
29725040,0irq/27-eth0-tx-0-21swapper/223:45:222
29725040,0irq/27-eth0-tx-0-21swapper/223:30:352
29725040,0irq/27-eth0-tx-0-21swapper/223:18:002
29725040,0irq/27-eth0-tx-0-21swapper/222:45:072
29725040,0irq/27-eth0-tx-0-21swapper/222:35:132
29725040,0irq/27-eth0-tx-0-21swapper/222:30:282
29725040,0irq/27-eth0-tx-0-21swapper/222:00:392
29725040,0irq/27-eth0-tx-0-21swapper/221:55:152
29725040,0irq/27-eth0-tx-0-21swapper/221:35:052
29725040,0irq/27-eth0-tx-0-21swapper/221:25:132
29725040,0irq/27-eth0-tx-0-21swapper/221:15:092
29725040,0irq/27-eth0-tx-0-21swapper/200:00:122
29725040,0irq/27-eth0-tx-0-21swapper/019:55:160
29715040,0irq/26-eth0-rx-0-21swapper/323:20:123
29715040,0irq/26-eth0-rx-0-21swapper/322:50:123
29715040,0irq/26-eth0-rx-0-21swapper/322:20:063
29715040,0irq/26-eth0-rx-0-21swapper/318:50:193
259940,0migration/2641-21kthreadcore21:45:222
259940,0migration/222864-21kthreadcore19:25:212
259940,0migration/220750-21aten_r1power_po23:20:152
259940,0migration/216982-21hwlatdetect20:10:192
259940,0migration/216982-21hwlatdetect20:10:192
249940,0watchdog/26559-21runrttasks20:25:272
249940,0watchdog/24619-21aten2.4-expect23:10:142
249940,0watchdog/223709-21kthreadcore23:40:222
249940,0watchdog/20-21swapper/223:35:222
249940,0watchdog/20-21swapper/218:55:232
179940,0migration/19386-21kthreadcore21:10:221
179940,0migration/17836-21kthreadcore22:50:211
179940,0migration/1351-21latency19:30:231
179940,0migration/132454-21kthreadcore23:45:211
179940,0migration/131945-21kthreadcore20:45:241
179940,0migration/131101-21kthreadcore20:20:211
179940,0migration/12953-21aten_r1power_vo20:00:171
179940,0migration/129389-21latency_hist21:25:041
179940,0migration/129149-21sh22:45:171
179940,0migration/127410-21unixbench-2d21:20:271
179940,0migration/126531-21kthreadcore20:15:221
179940,0migration/126490-21latency_hist19:05:051
179940,0migration/121882-21latency_hist19:50:051
179940,0migration/117990-21kthreadcore19:45:221
179940,0migration/117698-21kthreadcore22:35:231
179940,0migration/117355-21sh20:35:171
179940,0migration/113464-21fschecks_count20:55:211
179940,0migration/111947-21kthreadcore20:05:231
179940,0migration/110836-21cpuspeed_turbos20:30:171
179940,0migration/110081-21taskset20:27:521
169940,0watchdog/17621-21kthreadcore23:30:221
169940,0watchdog/16794-21kthreadcore19:35:231
169940,0watchdog/16001-21munin-node21:30:191
169940,0watchdog/15362-21diskstats21:50:151
169940,0watchdog/132704-21kthreadcore22:05:221
169940,0watchdog/128710-21ssh23:25:141
169940,0watchdog/127661-21sshd22:01:381
169940,0watchdog/12639-21ssh23:05:421
169940,0watchdog/111593-21ssh22:32:221
169940,0watchdog/10-21swapper/119:10:231
169940,0watchdog/10-21swapper/100:15:221
149940,0migration/04554-21kthreadcore20:00:220
149940,0migration/04070-21runrttasks23:49:570
149940,0migration/020195-21runrttasks21:37:590
63829932,0cyclictest32347-21timerwakeupswit19:05:302
63819932,0cyclictest8601-21grep22:10:211
63819932,0cyclictest7937-21df19:15:171
63819932,0cyclictest14121-21awk18:55:211
63819930,0cyclictest16593-21diskstats20:10:181
63819930,0cyclictest16593-21diskstats20:10:181
63809933,0cyclictest6617-21kworker/0:120:55:200
63809933,0cyclictest6617-21kworker/0:120:40:080
63809933,0cyclictest6617-21kworker/0:120:25:080
63809933,0cyclictest6617-21kworker/0:120:20:080
63809933,0cyclictest6617-21kworker/0:120:10:080
63809933,0cyclictest6617-21kworker/0:120:10:080
63809933,0cyclictest6617-21kworker/0:119:35:080
63809933,0cyclictest6617-21kworker/0:119:15:080
63809933,0cyclictest13713-21kworker/0:223:05:110
63809932,0cyclictest3514-21snmpd19:30:050
339930,0migration/326323-21runrttasks19:05:013
29715030,0irq/26-eth0-rx-0-21swapper/320:30:223
29715030,0irq/26-eth0-rx-0-21swapper/320:21:393
29715030,0irq/26-eth0-rx-0-21swapper/320:10:113
29715030,0irq/26-eth0-rx-0-21swapper/320:10:113
29715030,0irq/26-eth0-rx-0-21swapper/319:55:093
29715030,0irq/26-eth0-rx-0-21swapper/319:35:063
29715030,0irq/26-eth0-rx-0-21swapper/319:27:213
179930,0migration/129954-21runrttasks20:41:221
169930,0watchdog/10-21swapper/123:00:141
169930,0watchdog/10-21swapper/122:40:181
29715020,0irq/26-eth0-rx-0-21swapper/319:40:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional