You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 11:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Tue May 19, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2213024443,0sleep00-21swapper/018:52:440
2206023626,0sleep20-21swapper/218:51:582
304450350,0irq/25-eth10-21swapper/318:50:493
2228023122,0sleep10-21swapper/118:54:331
224129970,0cyclictest0-21swapper/122:46:571
224119976,0cyclictest0-21swapper/021:05:270
339960,0migration/330361-21unixbench-2d19:45:363
339960,0migration/33033-21sh22:43:263
339960,0migration/328370-21sh21:59:413
339960,0migration/327424-21latency_hist23:40:133
339960,0migration/32424-21sh23:22:583
339960,0migration/319397-21sh23:14:413
339960,0migration/318139-21sh23:52:143
339960,0migration/317462-21sh00:10:573
339960,0migration/310576-21sh00:09:213
29715060,0irq/26-eth0-rx-26080-21ssh23:15:513
29715060,0irq/26-eth0-rx-11026-21taskset21:06:033
259960,0migration/217822-21munin-node22:30:332
224139966,0cyclictest0-21swapper/221:54:562
149960,0migration/03718-21sh21:03:360
149960,0migration/02150-21sh00:24:040
149960,0migration/010526-21sh21:45:370
8215050,0irq/24-0000:00:18953-21sa100:15:132
35150,0ktimersoftd/30-21swapper/320:40:113
339950,0migration/39706-21switchtime23:05:373
339950,0migration/37274-21idleruntime20:20:293
339950,0migration/35322-21proc_pri20:15:343
339950,0migration/34792-21munin-node22:05:253
339950,0migration/329283-21irqcore23:00:323
339950,0migration/325641-21munin-node22:55:343
339950,0migration/324521-21kthreadcore21:15:303
339950,0migration/323819-21kthreadcore20:05:293
339950,0migration/32124-21rt-features22:00:363
339950,0migration/319529-21cpuspeed_turbos00:15:243
339950,0migration/31928-21sendmail_mailtr22:20:353
339950,0migration/318636-21munin-node19:40:293
339950,0migration/317504-21fschecks_time19:15:283
339950,0migration/313859-21taskset20:50:073
339950,0migration/311591-21sh21:47:393
339950,0migration/310599-21sh22:26:063
29725050,0irq/27-eth0-tx-608-21munin-node20:40:260
29725050,0irq/27-eth0-tx-5122-21kthreadcore23:25:292
29725050,0irq/27-eth0-tx-4456-21munin-node22:05:312
29725050,0irq/27-eth0-tx-3349-21munin-node23:45:292
29725050,0irq/27-eth0-tx-32413-21kthreadcore00:20:312
29725050,0irq/27-eth0-tx-3020-21ssh21:20:552
29725050,0irq/27-eth0-tx-25640-21python23:15:352
29725050,0irq/27-eth0-tx-12747-21if_err_eth1.223:10:272
29725050,0irq/27-eth0-tx-0-21swapper/223:02:572
29725050,0irq/27-eth0-tx-0-21swapper/222:02:292
29725050,0irq/27-eth0-tx-0-21swapper/200:02:152
29725050,0irq/27-eth0-tx-0-21swapper/020:05:230
29725050,0irq/27-eth0-tx-0-21swapper/020:00:100
29715050,0irq/26-eth0-rx-8300-21cut22:45:323
29715050,0irq/26-eth0-rx-6559-21runrttasks20:59:353
29715050,0irq/26-eth0-rx-30811-21kthreadcore00:00:313
29715050,0irq/26-eth0-rx-28261-21python18:55:353
29715050,0irq/26-eth0-rx-21650-21cut22:15:303
29715050,0irq/26-eth0-rx-15099-21cut23:30:343
29715050,0irq/26-eth0-rx-0-21swapper/323:55:443
29715050,0irq/26-eth0-rx-0-21swapper/323:49:393
29715050,0irq/26-eth0-rx-0-21swapper/322:50:303
29715050,0irq/26-eth0-rx-0-21swapper/322:35:253
29715050,0irq/26-eth0-rx-0-21swapper/321:50:333
29715050,0irq/26-eth0-rx-0-21swapper/321:35:293
29715050,0irq/26-eth0-rx-0-21swapper/321:12:413
29715050,0irq/26-eth0-rx-0-21swapper/321:00:333
29715050,0irq/26-eth0-rx-0-21swapper/320:10:323
29715050,0irq/26-eth0-rx-0-21swapper/300:23:373
27150,0ktimersoftd/20-21swapper/220:40:112
259950,0migration/25204-21munin-node20:15:332
259950,0migration/230152-21latency_hist19:25:142
259950,0migration/229663-21irqcore22:20:302
259950,0migration/224685-21needreboot00:15:312
259950,0migration/22245-21munin-node19:00:352
259950,0migration/217197-21munin-node23:30:362
259950,0migration/210524-21munin-node19:30:382
249950,0watchdog/29771-21processes22:45:362
249950,0watchdog/20-21swapper/223:55:312
224139955,0cyclictest0-21swapper/220:20:402
224139950,0cyclictest0-21swapper/223:08:342
224139950,0cyclictest0-21swapper/222:44:272
224129955,0cyclictest20534-21diskmemload22:35:071
224129955,0cyclictest0-21swapper/123:06:111
224129955,0cyclictest0-21swapper/122:01:231
224129955,0cyclictest0-21swapper/121:11:151
224129954,0cyclictest11143-21ssh22:07:031
224129950,0cyclictest0-21swapper/123:44:291
224129950,0cyclictest0-21swapper/121:35:411
224129950,0cyclictest0-21swapper/100:22:201
224119955,0cyclictest0-21swapper/023:23:340
224119950,0cyclictest0-21swapper/022:34:130
224119950,0cyclictest0-21swapper/021:33:030
179950,0migration/19673-21sh00:06:401
179950,0migration/127324-21sh22:58:461
179950,0migration/12644-21sh21:00:541
179950,0migration/118728-21taskset23:12:461
179950,0migration/118548-21kthreadcore19:15:311
179950,0migration/113683-21if_eth021:50:281
179950,0migration/11368-21sh00:01:141
179950,0migration/110704-21timerwakeupswit21:05:371
169950,0watchdog/117941-21missed_timers22:10:321
149950,0migration/09533-21sh00:06:160
149950,0migration/07337-21latency_hist20:45:140
149950,0migration/07168-21kthreadcore22:45:310
149950,0migration/06872-21kthreadcore22:25:290
149950,0migration/06414-21kthreadcore19:05:300
149950,0migration/031609-21kthreadcore23:00:330
149950,0migration/025888-21kthreadcore21:15:300
149950,0migration/023907-21taskset19:41:160
149950,0migration/022861-21kthreadcore00:15:300
149950,0migration/022173-21kthreadcore23:55:310
149950,0migration/021215-21kthreadcore20:30:310
149950,0migration/020230-21latency_hist22:35:140
149950,0migration/019882-21latency_hist22:55:130
149950,0migration/01836-21missed_timers22:00:330
149950,0migration/01692-21kthreadcore19:00:330
149950,0migration/013891-21munin-node20:25:280
149950,0migration/01194-21kthreadcore22:40:320
339940,0migration/37132-21taskset20:45:043
339940,0migration/35369-21latency_hist19:55:143
339940,0migration/35041-21kthreadcore19:05:303
339940,0migration/330390-21kthreadcore23:40:273
339940,0migration/324931-21kthreadcore19:20:343
339940,0migration/318458-21taskset20:04:173
339940,0migration/317565-21kthreadcore22:30:313
339940,0migration/316375-21taskset19:12:133
339940,0migration/315825-21kthreadcore20:25:313
339940,0migration/310241-21switchtime19:30:373
29725040,0irq/27-eth0-tx-20320-21munin-node22:35:282
29725040,0irq/27-eth0-tx-14697-21kthreadcore22:50:302
29725040,0irq/27-eth0-tx-0-21swapper/223:40:572
29725040,0irq/27-eth0-tx-0-21swapper/222:56:022
29725040,0irq/27-eth0-tx-0-21swapper/222:25:322
29725040,0irq/27-eth0-tx-0-21swapper/222:10:142
29725040,0irq/27-eth0-tx-0-21swapper/221:55:252
29725040,0irq/27-eth0-tx-0-21swapper/220:55:522
29725040,0irq/27-eth0-tx-0-21swapper/200:05:242
29725040,0irq/27-eth0-tx-0-21swapper/019:30:330
29715040,0irq/26-eth0-rx-0-21swapper/323:25:353
29715040,0irq/26-eth0-rx-0-21swapper/322:10:363
29715040,0irq/26-eth0-rx-0-21swapper/321:40:593
29715040,0irq/26-eth0-rx-0-21swapper/321:30:353
29715040,0irq/26-eth0-rx-0-21swapper/321:26:043
29715040,0irq/26-eth0-rx-0-21swapper/321:20:233
259940,0migration/25743-21irqcore21:45:312
259940,0migration/25139-21kthreadcore20:40:312
259940,0migration/232663-21kthreadcore21:40:302
259940,0migration/229521-21kthreadcore23:20:292
259940,0migration/227896-21munin-node18:55:312
259940,0migration/223196-21kthreadcore22:15:312
259940,0migration/222777-21kthreadcore20:30:312
259940,0migration/222745-21kthreadcore21:15:292
259940,0migration/217347-21fschecks_count19:15:282
259940,0migration/215011-21kthreadcore21:30:332
259940,0migration/214735-21kthreadcore23:50:312
259940,0migration/213223-21runrttasks20:45:392
249940,0watchdog/25854-21iostat_ios21:25:282
249940,0watchdog/225514-21basename20:10:242
249940,0watchdog/222648-21kthreadcore19:40:322
249940,0watchdog/211137-21sensors_volt19:55:362
249940,0watchdog/20-21swapper/221:05:162
249940,0watchdog/20-21swapper/220:05:282
249940,0watchdog/20-21swapper/219:35:282
224139943,0cyclictest3514-21snmpd21:10:222
224139943,0cyclictest3514-21snmpd21:00:222
224139943,0cyclictest29949-21sadc19:25:142
224139943,0cyclictest21781-21awk21:35:272
224139943,0cyclictest20514-21awk23:35:282
224129944,0cyclictest0-21swapper/123:51:351
224129944,0cyclictest0-21swapper/121:48:561
224129943,0cyclictest7160-21latency_hist20:45:141
224129943,0cyclictest4478-21awk23:25:281
224129943,0cyclictest3241-21irqbalance22:27:221
224129943,0cyclictest29895-21awk21:40:281
224129943,0cyclictest29302-21awk22:40:301
224129943,0cyclictest23602-21awk18:55:291
224129943,0cyclictest21707-21awk21:55:281
224129943,0cyclictest21270-21awk22:35:281
224129943,0cyclictest20688-21awk20:30:291
224129943,0cyclictest12506-21awk23:30:311
224129943,0cyclictest11152-21awk19:10:291
224129943,0cyclictest10950-21sadc19:35:141
224119944,0cyclictest0-21swapper/023:48:380
224119944,0cyclictest0-21swapper/023:13:090
224119944,0cyclictest0-21swapper/021:40:180
224119944,0cyclictest0-21swapper/020:10:260
224119943,0cyclictest5331-21awk22:05:270
224119943,0cyclictest4686-21awk23:05:290
224119943,0cyclictest3514-21snmpd20:35:220
224119943,0cyclictest32066-21awk19:50:290
224119943,0cyclictest29783-21kworker/0:122:22:370
224119943,0cyclictest25357-21awk19:45:280
224119943,0cyclictest13712-21awk21:50:290
224119943,0cyclictest11083-21awk19:10:280
179940,0migration/19646-21kthreadcore21:25:301
179940,0migration/16651-21munin-node19:05:301
179940,0migration/15769-21kthreadcore23:45:321
179940,0migration/132674-21kthreadcore19:00:331
179940,0migration/132665-21kthreadcore19:50:301
179940,0migration/131175-21kthreadcore22:20:311
179940,0migration/122896-21kthreadcore20:55:301
179940,0migration/118714-21munin-node19:40:301
179940,0migration/118306-21taskset22:50:511
179940,0migration/115810-21kthreadcore20:25:311
179940,0migration/115588-21kthreadcore19:35:321
179940,0migration/114537-21kthreadcore21:30:331
179940,0migration/113834-21kthreadcore20:00:291
179940,0migration/113070-21kthreadcore00:10:271
169940,0watchdog/127236-21ssh22:17:121
169940,0watchdog/12582-21ssh23:02:481
169940,0watchdog/124797-21kthreadcore23:35:311
169940,0watchdog/123787-21kthreadcore23:55:311
169940,0watchdog/10-21swapper/120:35:281
169940,0watchdog/10-21swapper/120:20:281
169940,0watchdog/10-21swapper/119:55:361
149940,0migration/08582-21kthreadcore20:20:320
149940,0migration/0771-21irqcore20:15:290
149940,0migration/03533-21sh23:25:220
149940,0migration/031194-21kthreadcore21:20:290
149940,0migration/02745-21sh00:05:110
149940,0migration/026165-21sh23:36:440
149940,0migration/024564-21kthreadcore18:55:300
149940,0migration/022121-21kthreadcore22:55:310
149940,0migration/019097-21kthreadcore20:50:320
149940,0migration/018627-21munin-node21:10:340
149940,0migration/016022-21kthreadcore19:35:320
8215030,0irq/24-0000:00:0-21swapper/219:10:142
29715030,0irq/26-eth0-rx-0-21swapper/320:54:073
29715030,0irq/26-eth0-rx-0-21swapper/319:35:303
29715030,0irq/26-eth0-rx-0-21swapper/319:29:483
224139932,0cyclictest3514-21snmpd20:50:142
224139932,0cyclictest3311-21kthreadcore19:50:312
224139932,0cyclictest15052-21grep20:00:302
224139930,0cyclictest13716-21perl20:25:272
224139930,0cyclictest0-21swapper/219:06:052
224129933,0cyclictest3556-21lldpd21:15:361
224129932,0cyclictest5420-21sensors20:15:351
224129932,0cyclictest3514-21snmpd19:24:581
224129932,0cyclictest32754-21kthreadcore19:25:331
224129932,0cyclictest28837-21awk23:20:271
224129932,0cyclictest25326-21awk19:45:271
224129932,0cyclictest19733-21awk20:05:261
224119933,0cyclictest32582-21kworker/0:020:00:250
224119933,0cyclictest32582-21kworker/0:019:20:300
224119933,0cyclictest32582-21kworker/0:019:15:260
224119933,0cyclictest29783-21kworker/0:123:40:190
224119933,0cyclictest29783-21kworker/0:123:30:290
224119933,0cyclictest29783-21kworker/0:122:15:150
224119933,0cyclictest29783-21kworker/0:122:10:250
224119933,0cyclictest29783-21kworker/0:121:55:190
224119933,0cyclictest2209-21kworker/0:021:35:390
224119933,0cyclictest2209-21kworker/0:021:25:330
224119933,0cyclictest13118-21kworker/0:120:55:150
224119933,0cyclictest1049-21kworker/0:223:50:210
224119933,0cyclictest1049-21kworker/0:200:10:180
224119932,0cyclictest25633-21sensors23:15:350
224119930,0cyclictest0-21swapper/019:25:290
169930,0watchdog/120130-21awk00:15:271
169930,0watchdog/119134-21kthreadcore20:50:321
169930,0watchdog/10-21swapper/123:15:311
169930,0watchdog/10-21swapper/121:20:241
29715020,0irq/26-eth0-rx-0-21swapper/320:30:333
29715020,0irq/26-eth0-rx-0-21swapper/319:50:533
29715020,0irq/26-eth0-rx-0-21swapper/319:00:183
249920,0watchdog/20-21swapper/219:45:202
224129921,0cyclictest7309-21cat20:45:141
224129920,0cyclictest0-21swapper/120:10:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional