You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 15:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Mar 15, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351560,0ktimersoftd/331468-21perf07:01:093
541023725,0sleep20-21swapper/206:58:102
544023524,0sleep00-21swapper/006:58:280
3168923228,0sleep10-21swapper/106:56:291
149970,0migration/010108-21sh10:02:460
339960,0migration/39871-21sh11:25:473
339960,0migration/331629-21munin-node11:56:373
339960,0migration/327621-21sh09:35:093
339960,0migration/326156-21sh10:54:433
339960,0migration/324174-21sh11:51:583
339960,0migration/310970-21sh09:03:403
29725060,0irq/27-eth0-tx-8932-21munin-node08:16:420
29715060,0irq/26-eth0-rx-4453-21if_err_eth1.210:01:323
29715060,0irq/26-eth0-rx-2727-21ssh09:59:493
149960,0migration/015515-21kthreadcore07:06:360
57559950,0cyclictest0-21swapper/210:45:042
57559950,0cyclictest0-21swapper/208:51:332
57539955,0cyclictest0-21swapper/112:01:241
57539955,0cyclictest0-21swapper/110:26:381
57539950,0cyclictest0-21swapper/111:56:211
57539950,0cyclictest0-21swapper/109:42:111
57539950,0cyclictest0-21swapper/109:23:531
57529955,0cyclictest0-21swapper/012:24:030
57529955,0cyclictest0-21swapper/007:21:200
57529950,0cyclictest0-21swapper/009:17:040
57529950,0cyclictest0-21swapper/009:10:070
57529950,0cyclictest0-21swapper/009:02:240
4150,0ktimersoftd/00-21swapper/011:03:210
4150,0ktimersoftd/00-21swapper/009:56:130
339950,0migration/34647-21munin-node09:21:303
339950,0migration/33812-21fschecks_time08:36:303
339950,0migration/33502-21kthreadcore11:41:343
339950,0migration/33440-21kthreadcore07:21:363
339950,0migration/329242-21kthreadcore10:56:333
339950,0migration/326822-21aten_r1power_po11:16:303
339950,0migration/325697-21smart_sdb09:51:403
339950,0migration/320418-21kthreadcore11:11:333
339950,0migration/318773-21kthreadcore07:31:363
339950,0migration/317465-21switchtime10:06:413
339950,0migration/315482-21kthreadcore09:06:353
339950,0migration/313448-21pmu-power07:51:393
339950,0migration/312120-21kthreadcore08:16:363
339950,0migration/31181-21kthreadcore08:56:353
339950,0migration/310098-21munin-node09:41:423
339950,0migration/310002-21fschecks_time12:26:313
29725050,0irq/27-eth0-tx-3914-21diskmemload12:02:442
29725050,0irq/27-eth0-tx-3221-21munin-node10:21:352
29725050,0irq/27-eth0-tx-20339-21munin-node09:11:342
29725050,0irq/27-eth0-tx-20320-21munin-node09:31:302
29725050,0irq/27-eth0-tx-19659-21munin-node10:11:382
29725050,0irq/27-eth0-tx-15926-21tr09:06:352
29725050,0irq/27-eth0-tx-15559-21munin-node08:21:380
29725050,0irq/27-eth0-tx-1244-21ssh11:19:092
29725050,0irq/27-eth0-tx-0-21swapper/211:25:412
29725050,0irq/27-eth0-tx-0-21swapper/209:40:272
29725050,0irq/27-eth0-tx-0-21swapper/007:29:410
29725050,0irq/27-eth0-tx-0-21swapper/007:12:410
29715050,0irq/26-eth0-rx-6513-21cut12:21:383
29715050,0irq/26-eth0-rx-22954-21kthreadcore11:31:363
29715050,0irq/26-eth0-rx-22240-21cut07:11:353
29715050,0irq/26-eth0-rx-18346-21munin-node09:26:403
29715050,0irq/26-eth0-rx-1598-21sh10:38:263
29715050,0irq/26-eth0-rx-15685-21processes11:46:393
29715050,0irq/26-eth0-rx-0-21swapper/312:11:313
29715050,0irq/26-eth0-rx-0-21swapper/312:08:003
29715050,0irq/26-eth0-rx-0-21swapper/311:38:293
29715050,0irq/26-eth0-rx-0-21swapper/311:26:323
29715050,0irq/26-eth0-rx-0-21swapper/311:07:363
29715050,0irq/26-eth0-rx-0-21swapper/311:01:283
29715050,0irq/26-eth0-rx-0-21swapper/310:42:013
29715050,0irq/26-eth0-rx-0-21swapper/310:22:323
29715050,0irq/26-eth0-rx-0-21swapper/310:11:393
29715050,0irq/26-eth0-rx-0-21swapper/309:47:103
29715050,0irq/26-eth0-rx-0-21swapper/309:41:163
29715050,0irq/26-eth0-rx-0-21swapper/309:16:493
259950,0migration/232278-21sh11:57:112
249950,0watchdog/20-21swapper/210:37:042
19150,0ktimersoftd/10-21swapper/111:03:211
179950,0migration/19203-21pmu-power10:01:381
179950,0migration/127641-21turbostat09:56:181
179950,0migration/125935-21cpuspeed_turbos12:16:321
179950,0migration/125766-21kthreadcore08:26:351
179950,0migration/118346-21rt-features07:06:391
179950,0migration/114160-21latency08:16:371
149950,0migration/09683-21rt-features09:41:380
149950,0migration/07432-21munin-node12:01:400
149950,0migration/06931-21munin-node07:46:410
149950,0migration/06261-21kthreadcore10:41:360
149950,0migration/05921-21kthreadcore11:21:360
149950,0migration/03448-21cpuspeed_turbos08:36:290
149950,0migration/032602-21kthreadcore10:16:370
149950,0migration/032540-21kthreadcore09:36:350
149950,0migration/03234-21aten_r1power_cu10:21:290
149950,0migration/031895-21kthreadcore11:16:360
149950,0migration/03096-21if_eth008:11:330
149950,0migration/030510-21fschecks_time08:56:320
149950,0migration/030196-21kthreadcore11:36:330
149950,0migration/02964-21taskset10:00:310
149950,0migration/027978-21kthreadcore08:51:360
149950,0migration/027746-21idleruntime07:41:320
149950,0migration/025123-21kthreadcore08:26:340
149950,0migration/024494-21kthreadcore09:11:350
149950,0migration/023781-21kthreadcore08:01:360
149950,0migration/020007-21kthreadcore12:11:340
149950,0migration/017360-21taskset11:50:170
149950,0migration/016529-21taskset08:46:240
149950,0migration/016439-21kthreadcore07:56:350
149950,0migration/015047-21missed_timers12:06:360
149950,0migration/014357-21kthreadcore08:41:360
149950,0migration/01291-21kthreadcore08:31:360
149950,0migration/012908-21kthreadcore11:06:350
149950,0migration/011976-21sh09:26:020
149950,0migration/01017-21sh10:57:400
57559944,0cyclictest0-21swapper/212:11:372
57559943,0cyclictest6855-21awk07:01:342
57559943,0cyclictest6855-21awk07:01:342
57559943,0cyclictest4044-21awk08:36:322
57559943,0cyclictest3514-21snmpd09:22:392
57559943,0cyclictest3514-21snmpd08:43:092
57559943,0cyclictest3514-21snmpd08:18:392
57559943,0cyclictest3514-21snmpd08:12:542
57559943,0cyclictest3514-21snmpd08:09:242
57559943,0cyclictest3514-21snmpd07:57:392
57559943,0cyclictest3514-21snmpd07:48:382
57559943,0cyclictest3514-21snmpd07:41:082
57559943,0cyclictest3514-21snmpd07:28:082
57559943,0cyclictest3241-21irqbalance11:08:172
57559943,0cyclictest3241-21irqbalance10:56:272
57559943,0cyclictest3241-21irqbalance09:41:272
57559943,0cyclictest30794-21awk08:56:332
57559943,0cyclictest29342-21awk09:16:332
57559943,0cyclictest29111-21irqrtprio09:56:342
57559943,0cyclictest19544-21awk11:11:312
57559943,0cyclictest17474-21awk08:46:342
57559943,0cyclictest12461-21awk10:06:332
57559943,0cyclictest10946-21awk11:46:332
57539944,0cyclictest3914-21diskmemload11:28:451
57539944,0cyclictest0-21swapper/111:40:251
57539944,0cyclictest0-21swapper/110:21:451
57539944,0cyclictest0-21swapper/109:47:511
57539943,0cyclictest3817-21awk10:41:331
57539943,0cyclictest3556-21lldpd10:34:481
57539943,0cyclictest3556-21lldpd10:19:181
57539943,0cyclictest3514-21snmpd09:27:091
57539943,0cyclictest3241-21irqbalance12:22:061
57539943,0cyclictest3241-21irqbalance08:51:271
57539943,0cyclictest3241-21irqbalance08:45:271
57539943,0cyclictest3241-21irqbalance08:36:271
57539943,0cyclictest3241-21irqbalance08:31:471
57539943,0cyclictest3241-21irqbalance08:21:371
57539943,0cyclictest3241-21irqbalance08:11:271
57539943,0cyclictest2840-21awk11:41:321
57539943,0cyclictest21020-21sadc08:01:181
57539943,0cyclictest20260-21awk07:11:331
57539943,0cyclictest19292-21awk11:31:341
57539943,0cyclictest14587-21awk07:31:331
57539943,0cyclictest11948-21awk10:46:341
57539943,0cyclictest10395-21grep12:06:311
57529944,0cyclictest0-21swapper/010:39:310
57529943,0cyclictest3556-21lldpd11:31:180
57529943,0cyclictest3556-21lldpd10:34:180
57529943,0cyclictest3556-21lldpd07:38:480
57529943,0cyclictest3514-21snmpd10:07:090
57529943,0cyclictest26940-21awk11:56:320
57529943,0cyclictest12817-21awk09:46:340
57529940,0cyclictest26030-21munin-node07:16:380
339940,0migration/331325-21kthreadcore07:41:353
339940,0migration/326689-21sh09:12:153
339940,0migration/324802-21kthreadcore07:36:363
339940,0migration/317972-21kthreadcore08:21:343
339940,0migration/316797-21kthreadcore07:06:363
339940,0migration/313295-21kthreadcore10:26:353
29725040,0irq/27-eth0-tx-18992-21munin-node10:51:392
29725040,0irq/27-eth0-tx-18543-21cpuspeed_turbos11:31:302
29725040,0irq/27-eth0-tx-0-21swapper/212:21:432
29725040,0irq/27-eth0-tx-0-21swapper/212:06:372
29725040,0irq/27-eth0-tx-0-21swapper/211:51:212
29725040,0irq/27-eth0-tx-0-21swapper/211:41:222
29725040,0irq/27-eth0-tx-0-21swapper/211:36:222
29725040,0irq/27-eth0-tx-0-21swapper/211:26:312
29725040,0irq/27-eth0-tx-0-21swapper/210:31:382
29725040,0irq/27-eth0-tx-0-21swapper/210:26:252
29725040,0irq/27-eth0-tx-0-21swapper/210:02:472
29725040,0irq/27-eth0-tx-0-21swapper/209:52:362
29725040,0irq/27-eth0-tx-0-21swapper/209:02:442
29725040,0irq/27-eth0-tx-0-21swapper/007:51:300
29715040,0irq/26-eth0-rx-0-21swapper/312:20:333
29715040,0irq/26-eth0-rx-0-21swapper/312:01:223
29715040,0irq/26-eth0-rx-0-21swapper/310:46:303
29715040,0irq/26-eth0-rx-0-21swapper/310:31:343
29715040,0irq/26-eth0-rx-0-21swapper/310:16:363
29715040,0irq/26-eth0-rx-0-21swapper/308:11:383
29715040,0irq/26-eth0-rx-0-21swapper/308:06:333
29715040,0irq/26-eth0-rx-0-21swapper/307:56:313
29715040,0irq/26-eth0-rx-0-21swapper/307:46:313
29715040,0irq/26-eth0-rx-0-21swapper/307:16:363
29715040,0irq/26-eth0-rx-0-21swapper/307:01:313
29715040,0irq/26-eth0-rx-0-21swapper/307:01:313
259940,0migration/226861-21irqrtprio12:16:362
259940,0migration/222359-21kthreadcore07:11:352
259940,0migration/217524-21kthreadcore09:26:352
249940,0watchdog/28738-21munin-node07:51:332
249940,0watchdog/231353-21cut10:16:372
249940,0watchdog/216562-21iostat_ios08:21:332
249940,0watchdog/216052-21kthreadcore10:46:362
249940,0watchdog/20-21swapper/212:27:242
249940,0watchdog/20-21swapper/211:01:362
249940,0watchdog/20-21swapper/209:46:292
179940,0migration/17953-21kthreadcore07:01:361
179940,0migration/17953-21kthreadcore07:01:361
179940,0migration/16789-21kthreadcore11:21:371
179940,0migration/15643-21missed_timers07:21:371
179940,0migration/14398-21cpuspeed_turbos09:01:301
179940,0migration/127432-21fschecks_count10:56:291
179940,0migration/127196-21latency_hist10:36:181
179940,0migration/127096-21runrttasks07:41:301
179940,0migration/126247-21sh10:13:041
179940,0migration/126090-21aten2.4-expect07:16:291
179940,0migration/125264-21kthreadcore07:36:361
179940,0migration/122471-21kthreadcore09:11:341
179940,0migration/121631-21kthreadcore10:51:341
179940,0migration/121145-21idleruntime09:31:321
179940,0migration/120872-21kthreadcore08:46:371
179940,0migration/120448-21kthreadcore11:51:341
179940,0migration/11996-21munin-node09:36:431
179940,0migration/119010-21taskset09:08:201
179940,0migration/116127-21kthreadcore10:06:361
179940,0migration/115295-21kthreadcore11:06:361
179940,0migration/114495-21latency_hist07:56:181
179940,0migration/11016-21kthreadcore08:56:351
149940,0migration/04736-21kthreadcore11:41:340
149940,0migration/029762-21kthreadcore12:16:370
149940,0migration/016985-21proc_pri10:26:390
149940,0migration/014330-21kthreadcore09:26:340
149940,0migration/012636-21kthreadcore11:26:340
57559932,0cyclictest31697-21smartctl07:16:402
57559932,0cyclictest30670-21kthreadcore08:31:352
57559932,0cyclictest23048-21awk08:26:322
57559932,0cyclictest14518-21awk07:31:322
57559932,0cyclictest13587-21awk07:06:342
57539932,0cyclictest3514-21snmpd09:56:391
57539932,0cyclictest28783-21cut09:16:301
57539932,0cyclictest18494-21awk12:11:321
57539932,0cyclictest13775-21cut12:26:351
57539931,0cyclictest6902-21gzip07:46:401
57539931,0cyclictest191ktimersoftd/107:26:361
57529933,0cyclictest9029-21kworker/0:007:31:200
57529933,0cyclictest9029-21kworker/0:007:01:270
57529933,0cyclictest9029-21kworker/0:007:01:270
57529933,0cyclictest27881-21kworker/0:010:51:240
57529933,0cyclictest27881-21kworker/0:010:46:440
57529933,0cyclictest27881-21kworker/0:009:31:540
57529933,0cyclictest2342-21kworker/0:212:26:290
57529933,0cyclictest22037-21kworker/0:108:06:220
57529933,0cyclictest19404-21kworker/0:111:11:460
57529932,0cyclictest20963-21uname10:11:350
57529932,0cyclictest0-21swapper/011:51:340
29715030,0irq/26-eth0-rx-0-21swapper/308:51:413
29715030,0irq/26-eth0-rx-0-21swapper/308:46:313
29715030,0irq/26-eth0-rx-0-21swapper/308:42:453
29715030,0irq/26-eth0-rx-0-21swapper/308:28:413
29715030,0irq/26-eth0-rx-0-21swapper/308:01:313
29715030,0irq/26-eth0-rx-0-21swapper/307:29:513
259930,0migration/21780-21kthreadcore07:21:362
249930,0watchdog/20-21swapper/208:01:332
249930,0watchdog/20-21swapper/207:44:492
169930,0watchdog/10-21swapper/111:46:361
169930,0watchdog/10-21swapper/111:16:201
169930,0watchdog/10-21swapper/111:13:281
169930,0watchdog/10-21swapper/108:06:331
169930,0watchdog/10-21swapper/108:01:371
29715020,0irq/26-eth0-rx-0-21swapper/308:31:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional