You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-15 - 12:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Mar 15, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450640,0irq/25-eth10-21swapper/318:56:323
2539523322,0sleep00-21swapper/018:59:440
2550323127,0sleep20-21swapper/219:01:022
2514322622,0sleep10-21swapper/118:56:491
25605991212,0cyclictest3514-21snmpd00:06:401
339960,0migration/32204-21munin-node19:56:323
339960,0migration/321344-21sh00:00:453
339960,0migration/314690-21sh21:13:393
29725060,0irq/27-eth0-tx-29596-21munin-node20:41:410
29715060,0irq/26-eth0-rx-9381-21grep22:31:353
29715060,0irq/26-eth0-rx-23861-21cat21:01:213
29715060,0irq/26-eth0-rx-21304-21sh23:39:503
149960,0migration/08675-21latency_hist20:01:220
149960,0migration/06264-21sh21:47:480
149960,0migration/06102-21sh23:11:090
149960,0migration/030264-21sh23:05:370
149960,0migration/022080-21sh23:41:280
149960,0migration/02030-21irqcore19:31:370
149960,0migration/014121-21sh22:13:160
339950,0migration/39216-21fschecks_count20:01:343
339950,0migration/36718-21latency_hist19:11:223
339950,0migration/35906-21munin-node21:46:473
339950,0migration/34918-21latency_hist00:31:213
339950,0migration/33374-21munin-node23:46:413
339950,0migration/332601-21kthreadcore20:41:383
339950,0migration/330657-21idleruntime00:06:363
339950,0migration/327735-21switchtime20:11:423
339950,0migration/327735-21switchtime20:11:423
339950,0migration/326017-21kthreadcore21:21:373
339950,0migration/325274-21kthreadcore22:21:373
339950,0migration/319978-21kthreadcore21:56:383
339950,0migration/319815-21kthreadcore20:56:373
339950,0migration/319598-21kthreadcore22:16:393
339950,0migration/317161-21kthreadcore23:16:373
339950,0migration/316802-21gltestperf21:16:363
339950,0migration/316049-21munin-node20:51:413
339950,0migration/316010-21kthreadcore19:41:373
339950,0migration/315720-21kthreadcore00:16:383
339950,0migration/31361-21kthreadcore19:06:403
339950,0migration/310844-21kthreadcore22:11:373
29725050,0irq/27-eth0-tx-24356-21ssh21:01:340
29725050,0irq/27-eth0-tx-0-21swapper/221:39:022
29725050,0irq/27-eth0-tx-0-21swapper/221:35:002
29725050,0irq/27-eth0-tx-0-21swapper/019:36:340
29715050,0irq/26-eth0-rx-31745-21sh22:06:263
29715050,0irq/26-eth0-rx-31259-21idleruntime23:26:353
29715050,0irq/26-eth0-rx-28263-21grep19:51:343
29715050,0irq/26-eth0-rx-27361-21kthreadcore22:01:383
29715050,0irq/26-eth0-rx-27260-21kthreadcore23:41:393
29715050,0irq/26-eth0-rx-22324-21fschecks_count00:01:343
29715050,0irq/26-eth0-rx-20464-21port_sshpriv22:56:423
29715050,0irq/26-eth0-rx-17056-21if_eth121:36:363
29715050,0irq/26-eth0-rx-12009-21kthreadcore21:51:383
29715050,0irq/26-eth0-rx-11783-21sh00:12:043
29715050,0irq/26-eth0-rx-0-21swapper/323:51:453
29715050,0irq/26-eth0-rx-0-21swapper/323:25:013
29715050,0irq/26-eth0-rx-0-21swapper/323:08:053
29715050,0irq/26-eth0-rx-0-21swapper/323:02:133
29715050,0irq/26-eth0-rx-0-21swapper/322:52:463
29715050,0irq/26-eth0-rx-0-21swapper/322:46:363
29715050,0irq/26-eth0-rx-0-21swapper/322:37:473
29715050,0irq/26-eth0-rx-0-21swapper/321:32:463
29715050,0irq/26-eth0-rx-0-21swapper/321:28:103
256059955,0cyclictest6559-21runrttasks23:14:401
256059955,0cyclictest0-21swapper/121:55:411
256049955,0cyclictest0-21swapper/023:51:330
256049955,0cyclictest0-21swapper/023:30:530
256049955,0cyclictest0-21swapper/023:24:160
256049955,0cyclictest0-21swapper/000:24:010
256049954,0cyclictest14827-21crond22:36:210
259950,0migration/225007-21kthreadcore23:01:392
249950,0watchdog/23066-21kthreadcore19:06:402
179950,0migration/130943-21kthreadcore00:26:371
179950,0migration/128520-21taskset00:24:181
179950,0migration/12416-21kthreadcore19:06:401
179950,0migration/123863-21aten_r1power_po22:01:321
179950,0migration/122639-21munin-node20:56:391
179950,0migration/114116-21taskset19:41:011
179950,0migration/114035-21fschecks_count00:16:341
179950,0migration/113569-21munin-node21:11:401
149950,0migration/09663-21aten_r1power_cu20:26:320
149950,0migration/08869-21irqcore21:51:370
149950,0migration/08847-21kthreadcore19:11:380
149950,0migration/07487-21kthreadcore23:31:380
149950,0migration/0639-21kernelversion22:06:390
149950,0migration/032164-21kthreadcore23:46:370
149950,0migration/030111-21fschecks_time00:26:340
149950,0migration/028962-21cpuspeed_turbos20:16:340
149950,0migration/026367-21kthreadcore22:01:370
149950,0migration/026320-21kthreadcore22:21:370
149950,0migration/025660-21kthreadcore00:01:380
149950,0migration/022605-21taskset22:40:480
149950,0migration/021954-21switchtime21:36:430
149950,0migration/021929-21latency_hist20:11:220
149950,0migration/021929-21latency_hist20:11:220
149950,0migration/020860-21sh23:17:310
149950,0migration/020707-21missed_timers20:06:410
149950,0migration/020145-21timerandwakeup19:41:430
149950,0migration/017670-21kthreadcore19:16:400
149950,0migration/01667-21taskset19:55:290
149950,0migration/015928-21latency_hist21:36:210
149950,0migration/015707-21cpuspeed_turbos22:16:330
149950,0migration/015620-21kthreadcore00:16:380
149950,0migration/01516-21kthreadcore22:26:380
149950,0migration/012366-21runrttasks00:13:460
8215040,0irq/24-0000:00:0-21swapper/222:46:212
339940,0migration/34294-21kthreadcore19:31:383
339940,0migration/331617-21cpu22:26:333
339940,0migration/325848-21switchtime19:21:443
339940,0migration/324424-21kthreadcore00:21:383
29725040,0irq/27-eth0-tx-31620-21sshd23:06:392
29725040,0irq/27-eth0-tx-31448-21ssh22:26:322
29725040,0irq/27-eth0-tx-26796-21telnet19:26:320
29725040,0irq/27-eth0-tx-26711-21kthreadcore22:01:372
29725040,0irq/27-eth0-tx-23711-21diskmemload22:41:212
29725040,0irq/27-eth0-tx-23711-21diskmemload21:56:392
29725040,0irq/27-eth0-tx-20191-21kthreadcore22:36:382
29725040,0irq/27-eth0-tx-20075-21pmu-power23:16:412
29725040,0irq/27-eth0-tx-1981-21munin-node19:56:380
29725040,0irq/27-eth0-tx-0-21swapper/223:56:392
29725040,0irq/27-eth0-tx-0-21swapper/223:51:272
29725040,0irq/27-eth0-tx-0-21swapper/223:36:592
29725040,0irq/27-eth0-tx-0-21swapper/223:32:052
29725040,0irq/27-eth0-tx-0-21swapper/223:26:232
29725040,0irq/27-eth0-tx-0-21swapper/223:21:262
29725040,0irq/27-eth0-tx-0-21swapper/222:58:482
29725040,0irq/27-eth0-tx-0-21swapper/222:51:252
29725040,0irq/27-eth0-tx-0-21swapper/222:16:312
29725040,0irq/27-eth0-tx-0-21swapper/222:11:572
29725040,0irq/27-eth0-tx-0-21swapper/222:06:492
29725040,0irq/27-eth0-tx-0-21swapper/221:26:422
29725040,0irq/27-eth0-tx-0-21swapper/200:26:262
29725040,0irq/27-eth0-tx-0-21swapper/200:23:502
29725040,0irq/27-eth0-tx-0-21swapper/200:17:402
29725040,0irq/27-eth0-tx-0-21swapper/200:11:372
29725040,0irq/27-eth0-tx-0-21swapper/200:01:362
29725040,0irq/27-eth0-tx-0-21swapper/020:51:420
29725040,0irq/27-eth0-tx-0-21swapper/020:21:370
29725040,0irq/27-eth0-tx-0-21swapper/019:24:410
29715040,0irq/26-eth0-rx-6559-21runrttasks19:16:393
29715040,0irq/26-eth0-rx-0-21swapper/323:31:233
29715040,0irq/26-eth0-rx-0-21swapper/323:11:323
29715040,0irq/26-eth0-rx-0-21swapper/322:41:263
29715040,0irq/26-eth0-rx-0-21swapper/321:41:323
29715040,0irq/26-eth0-rx-0-21swapper/321:08:043
29715040,0irq/26-eth0-rx-0-21swapper/320:06:403
29715040,0irq/26-eth0-rx-0-21swapper/319:46:443
29715040,0irq/26-eth0-rx-0-21swapper/319:26:353
256069943,0cyclictest3514-21snmpd19:36:232
256069943,0cyclictest3241-21irqbalance23:41:392
256069943,0cyclictest3241-21irqbalance22:21:402
256069943,0cyclictest3241-21irqbalance21:51:402
256069943,0cyclictest3241-21irqbalance19:41:402
256069943,0cyclictest3241-21irqbalance19:17:102
256069943,0cyclictest3241-21irqbalance19:14:102
256069943,0cyclictest3241-21irqbalance19:01:402
256069943,0cyclictest31112-21awk23:46:352
256069943,0cyclictest24878-21awk21:01:362
256069943,0cyclictest16925-21awk21:16:372
256069940,0cyclictest0-21swapper/200:07:472
256059943,0cyclictest8332-21awk22:11:341
256059943,0cyclictest7686-21awk22:51:351
256059943,0cyclictest6948-21awk23:31:351
256059943,0cyclictest4857-21awk20:46:361
256059943,0cyclictest3514-21snmpd23:16:231
256059943,0cyclictest3514-21snmpd20:26:381
256059943,0cyclictest3241-21irqbalance23:36:391
256059943,0cyclictest3241-21irqbalance22:39:091
256059943,0cyclictest3241-21irqbalance21:37:001
256059943,0cyclictest3241-21irqbalance21:16:401
256059943,0cyclictest31699-21awk23:06:391
256059943,0cyclictest26668-21awk19:01:371
256059943,0cyclictest24902-21awk21:01:361
256059943,0cyclictest16309-21awk22:16:361
256049944,0cyclictest22644-21kworker/0:021:26:530
256049943,0cyclictest933-21awk19:06:380
256049943,0cyclictest3556-21lldpd21:19:150
256049943,0cyclictest3241-21irqbalance23:11:290
256049943,0cyclictest3241-21irqbalance21:42:300
256049943,0cyclictest26770-21awk19:01:370
256049943,0cyclictest23864-21awk20:36:360
256049943,0cyclictest21549-21sadc00:01:210
256049943,0cyclictest15792-21awk22:56:360
259940,0migration/29088-21kthreadcore20:46:382
259940,0migration/28756-21if_eth121:11:342
259940,0migration/24365-21kthreadcore19:56:372
259940,0migration/24092-21kthreadcore21:06:392
259940,0migration/231175-21kthreadcore19:26:392
259940,0migration/225320-21kthreadcore19:21:402
259940,0migration/224182-21latency_hist21:21:212
259940,0migration/222853-21kthreadcore19:46:372
259940,0migration/219767-21kthreadcore20:56:372
259940,0migration/216451-21cpu20:31:332
259940,0migration/21309-21df19:31:332
259940,0migration/211312-21kthreadcore20:26:382
259940,0migration/211212-21kthreadcore20:01:382
249940,0watchdog/28925-21kthreadcore22:31:352
249940,0watchdog/26535-21munin-node23:11:312
249940,0watchdog/222228-21cpu20:11:322
249940,0watchdog/222228-21cpu20:11:322
249940,0watchdog/21254-21taskset19:51:442
249940,0watchdog/20-21swapper/221:46:402
249940,0watchdog/20-21swapper/221:41:442
249940,0watchdog/20-21swapper/220:41:442
179940,0migration/19230-21kthreadcore00:11:391
179940,0migration/18822-21fschecks_time21:31:341
179940,0migration/16780-21aten_r1power_cu19:11:321
179940,0migration/15989-21kthreadcore19:56:381
179940,0migration/14682-21runrttasks23:28:091
179940,0migration/14659-21kthreadcore20:21:381
179940,0migration/14149-21kthreadcore22:26:391
179940,0migration/131575-21kthreadcore19:26:391
179940,0migration/129106-21taskset00:05:041
179940,0migration/129010-21taskset20:37:211
179940,0migration/128369-21kthreadcore21:41:381
179940,0migration/124088-21fschecks_time22:21:341
179940,0migration/121066-21munin-node19:21:381
179940,0migration/119511-21runrttasks19:17:451
179940,0migration/117397-21kthreadcore20:06:391
179940,0migration/116181-21df_abs21:56:331
179940,0migration/113751-21latency_hist23:56:211
169940,0watchdog/17055-21rm22:31:231
169940,0watchdog/15400-21python21:06:441
169940,0watchdog/13092-21unixbench-2d20:41:441
169940,0watchdog/129349-21ssh22:41:591
169940,0watchdog/12851-21kthreadcore21:46:401
169940,0watchdog/127928-21swap23:41:431
169940,0watchdog/125403-21kthreadcore23:01:391
169940,0watchdog/10-21swapper/123:51:351
149940,0migration/09587-21kthreadcore21:11:350
149940,0migration/03347-21kthreadcore22:46:360
149940,0migration/031218-21kthreadcore00:06:380
149940,0migration/029000-21switchtime22:41:450
149940,0migration/02015-21kthreadcore21:06:380
149940,0migration/017908-21kthreadcore23:36:390
149940,0migration/017421-21irqcore20:31:370
339930,0migration/322696-21runrttasks20:36:083
29715030,0irq/26-eth0-rx-0-21swapper/320:46:323
29715030,0irq/26-eth0-rx-0-21swapper/320:26:323
29715030,0irq/26-eth0-rx-0-21swapper/320:21:333
29715030,0irq/26-eth0-rx-0-21swapper/319:37:003
29455230,0sleep30-21swapper/320:16:373
256069932,0cyclictest11522-21awk20:51:362
256069930,0cyclictest0-21swapper/220:25:292
256059933,0cyclictest25254-21awk21:21:361
256059932,0cyclictest3981-21grep19:31:381
256059932,0cyclictest20045-21sensors19:41:421
256059932,0cyclictest15742-21awk22:56:361
256059932,0cyclictest11546-21awk20:51:371
256059930,0cyclictest22536-21aten2.4-expect23:21:321
256049933,0cyclictest32573-21kworker/0:222:51:260
256049933,0cyclictest22644-21kworker/0:021:56:230
256049933,0cyclictest22644-21kworker/0:021:21:270
256049933,0cyclictest22644-21kworker/0:020:56:390
256049933,0cyclictest22644-21kworker/0:020:46:230
256049933,0cyclictest22644-21kworker/0:019:46:330
249930,0watchdog/20-21swapper/220:36:402
249930,0watchdog/20-21swapper/220:06:242
179930,0migration/128006-21cpuspeed_turbos19:51:331
179930,0migration/122510-21runrttasks20:33:581
169930,0watchdog/131521-21ssh22:46:311
169930,0watchdog/123711-21diskmemload23:47:511
169930,0watchdog/123711-21diskmemload21:26:361
169930,0watchdog/10-21swapper/122:06:391
169930,0watchdog/10-21swapper/120:11:321
169930,0watchdog/10-21swapper/120:11:321
329920,0watchdog/30-21swapper/320:36:373
29715020,0irq/26-eth0-rx-0-21swapper/319:01:373
249920,0watchdog/20-21swapper/220:16:282
169920,0watchdog/10-21swapper/120:16:241
169920,0watchdog/10-21swapper/120:01:241
169920,0watchdog/10-21swapper/119:46:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional