You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 18:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Jul 08, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351860,0ktimersoftd/315253-21perf06:50:153
2156623423,0sleep20-21swapper/206:47:352
2153723423,0sleep00-21swapper/006:47:170
2152323324,0sleep10-21swapper/106:47:061
2188299140,0cyclictest0-21swapper/008:52:430
218829970,0cyclictest9455-21ssh11:01:240
339960,0migration/36640-21fschecks_time08:15:383
339960,0migration/326055-21sh09:51:023
339960,0migration/32270-21sh11:19:273
339960,0migration/318446-21sh11:29:273
339960,0migration/317548-21sh10:46:073
339960,0migration/310411-21sh10:02:463
29715060,0irq/26-eth0-rx-9698-21ntp_states09:00:443
29715060,0irq/26-eth0-rx-4604-21munin-node09:20:363
29715060,0irq/26-eth0-rx-16495-21cut11:45:443
29715060,0irq/26-eth0-rx-13213-21cut09:25:413
218859960,0cyclictest0-21swapper/210:05:022
19160,0ktimersoftd/10-21swapper/108:52:481
149960,0migration/02629-21sh09:58:460
149960,0migration/023825-21ntp_offset08:00:440
149960,0migration/021843-21kthreadcore10:30:410
149960,0migration/01335-21sh11:57:320
149960,0migration/01180-21pmu-power09:35:450
339950,0migration/36906-21kthreadcore11:00:413
339950,0migration/35914-21runrttasks08:35:473
339950,0migration/34548-21fschecks_count07:25:393
339950,0migration/33608-21munin-node12:00:393
339950,0migration/332763-21munin-node12:15:463
339950,0migration/332411-21gltestperf08:10:363
339950,0migration/331693-21munin-node07:45:403
339950,0migration/328966-21hwlatdetect08:55:403
339950,0migration/327932-21aten2.4-expect09:55:343
339950,0migration/327234-21kernelversion08:30:413
339950,0migration/325624-21taskset09:30:453
339950,0migration/321380-21kthreadcore11:30:403
339950,0migration/320268-21munin-node08:50:323
339950,0migration/319367-21latency_hist10:30:243
339950,0migration/318670-21taskset10:08:253
339950,0migration/318297-21runrttasks11:08:403
339950,0migration/317353-21runrttasks12:07:083
339950,0migration/313372-21taskset08:45:203
339950,0migration/312533-21irqcore10:25:393
339950,0migration/311802-21munin-node07:55:363
29725050,0irq/27-eth0-tx-29201-21irqrtprio09:35:412
29725050,0irq/27-eth0-tx-28769-21munin-node06:55:490
29725050,0irq/27-eth0-tx-28640-21if_eth1.310:15:382
29725050,0irq/27-eth0-tx-25957-21munin-node08:30:370
29725050,0irq/27-eth0-tx-19778-21munin-node09:50:442
29725050,0irq/27-eth0-tx-19334-21munin-node10:50:452
29725050,0irq/27-eth0-tx-18256-21ssh10:07:152
29725050,0irq/27-eth0-tx-1421-21ssh11:57:472
29725050,0irq/27-eth0-tx-10742-21munin-node12:05:372
29725050,0irq/27-eth0-tx-0-21swapper/211:26:072
29725050,0irq/27-eth0-tx-0-21swapper/210:55:452
29725050,0irq/27-eth0-tx-0-21swapper/210:46:202
29725050,0irq/27-eth0-tx-0-21swapper/209:25:532
29725050,0irq/27-eth0-tx-0-21swapper/007:20:510
29715050,0irq/26-eth0-rx-3906-21cpuspeed_turbos09:40:363
29715050,0irq/26-eth0-rx-30291-21kthreadcore09:35:423
29715050,0irq/26-eth0-rx-2668-21ssh10:19:253
29715050,0irq/26-eth0-rx-23907-21proc_pri08:00:453
29715050,0irq/26-eth0-rx-19852-21munin-node11:50:383
29715050,0irq/26-eth0-rx-0-21swapper/311:40:373
29715050,0irq/26-eth0-rx-0-21swapper/311:39:323
29715050,0irq/26-eth0-rx-0-21swapper/311:12:093
29715050,0irq/26-eth0-rx-0-21swapper/310:41:073
29715050,0irq/26-eth0-rx-0-21swapper/310:36:233
29715050,0irq/26-eth0-rx-0-21swapper/310:20:353
29715050,0irq/26-eth0-rx-0-21swapper/310:10:403
29715050,0irq/26-eth0-rx-0-21swapper/309:10:383
29715050,0irq/26-eth0-rx-0-21swapper/308:20:393
259950,0migration/29504-21sh11:42:162
259950,0migration/25453-21kthreadcore10:20:412
259950,0migration/222126-21switchtime07:10:482
259950,0migration/218988-21latency_hist11:30:242
249950,0watchdog/22110-21kthreadcore07:20:432
218859955,0cyclictest0-21swapper/209:33:402
218859954,0cyclictest18227-21awk07:35:412
218859950,0cyclictest0-21swapper/212:15:542
218859950,0cyclictest0-21swapper/211:11:532
218859950,0cyclictest0-21swapper/211:02:352
218859950,0cyclictest0-21swapper/209:16:102
218839955,0cyclictest0-21swapper/110:48:001
218839955,0cyclictest0-21swapper/107:45:451
218839950,0cyclictest0-21swapper/111:45:431
218839950,0cyclictest0-21swapper/111:19:231
218829950,0cyclictest0-21swapper/009:35:000
218829950,0cyclictest0-21swapper/009:04:440
19150,0ktimersoftd/10-21swapper/109:11:031
179950,0migration/18001-21kthreadcore07:50:421
179950,0migration/15925-21munin-node08:35:471
179950,0migration/132275-21kthreadcore09:55:411
179950,0migration/127925-21cpuspeed_turbos10:15:341
179950,0migration/116839-21fschecks_count07:10:381
179950,0migration/114876-21irqrtprio08:45:401
179950,0migration/114876-21irqrtprio08:45:401
179950,0migration/114273-21kthreadcore07:30:421
179950,0migration/113688-21kthreadcore12:05:401
179950,0migration/111488-21kthreadcore07:05:421
179950,0migration/111474-21sh09:24:581
169950,0watchdog/130453-21gltestperf07:20:391
149950,0migration/09305-21timerandwakeup10:20:470
149950,0migration/08593-21latency10:40:420
149950,0migration/0600-21fschecks_count08:35:390
149950,0migration/031311-21kthreadcore10:15:400
149950,0migration/030493-21kthreadcore08:55:430
149950,0migration/030252-21latency08:05:410
149950,0migration/029349-21kthreadcore10:35:410
149950,0migration/029207-21kthreadcore11:35:420
149950,0migration/027956-21fschecks_time10:55:370
149950,0migration/027547-21taskset09:55:170
149950,0migration/026950-21latency_hist11:35:240
149950,0migration/024366-21kthreadcore10:10:400
149950,0migration/023462-21kthreadcore11:50:400
149950,0migration/022651-21kthreadcore09:10:420
149950,0migration/021806-21kthreadcore08:25:420
149950,0migration/020965-21kthreadcore11:10:390
149950,0migration/016933-21smart_sdb11:05:460
149950,0migration/016182-21kthreadcore10:25:410
149950,0migration/013810-21munin-node08:20:400
149950,0migration/013534-21kthreadcore10:45:420
149950,0migration/012645-21kthreadcore07:05:430
149950,0migration/012431-21kthreadcore12:05:400
149950,0migration/012277-21missed_timers08:40:440
149950,0migration/010489-21runrttasks07:50:450
149950,0migration/010430-21kthreadcore08:15:420
339940,0migration/33635-21fschecks_time11:20:383
339940,0migration/323623-21taskset07:38:443
29725040,0irq/27-eth0-tx-3003-21munin-node07:00:430
29725040,0irq/27-eth0-tx-27535-21munin-node11:35:382
29725040,0irq/27-eth0-tx-2613-21ssh12:00:312
29725040,0irq/27-eth0-tx-18787-21munin-node12:10:422
29725040,0irq/27-eth0-tx-0-21swapper/211:15:302
29725040,0irq/27-eth0-tx-0-21swapper/211:05:382
29725040,0irq/27-eth0-tx-0-21swapper/210:40:402
29725040,0irq/27-eth0-tx-0-21swapper/210:30:572
29725040,0irq/27-eth0-tx-0-21swapper/210:26:092
29725040,0irq/27-eth0-tx-0-21swapper/209:48:032
29725040,0irq/27-eth0-tx-0-21swapper/209:10:282
29725040,0irq/27-eth0-tx-0-21swapper/209:05:362
29725040,0irq/27-eth0-tx-0-21swapper/007:45:370
29715040,0irq/26-eth0-rx-0-21swapper/312:12:203
29715040,0irq/26-eth0-rx-0-21swapper/311:56:333
29715040,0irq/26-eth0-rx-0-21swapper/310:56:053
29715040,0irq/26-eth0-rx-0-21swapper/310:52:243
29715040,0irq/26-eth0-rx-0-21swapper/309:46:443
29715040,0irq/26-eth0-rx-0-21swapper/309:15:443
29715040,0irq/26-eth0-rx-0-21swapper/309:05:493
29715040,0irq/26-eth0-rx-0-21swapper/307:05:463
29715040,0irq/26-eth0-rx-0-21swapper/306:55:483
259940,0migration/27037-21kthreadcore07:00:422
259940,0migration/23715-21aten_r1power_en09:40:352
259940,0migration/230565-21kthreadcore10:35:422
259940,0migration/219842-21if_eth011:50:382
259940,0migration/21913-21sh08:55:522
259940,0migration/21900-21kthreadcore08:10:402
259940,0migration/214219-21kthreadcore08:20:412
259940,0migration/212100-21kthreadcore07:05:432
249940,0watchdog/29982-21cut07:50:432
249940,0watchdog/24111-21ssh09:00:352
249940,0watchdog/22859-21ssh11:20:262
249940,0watchdog/225176-21aten2.4-expect08:05:352
249940,0watchdog/224969-21python08:25:472
249940,0watchdog/219855-21cpu10:10:352
249940,0watchdog/20-21swapper/208:40:392
249940,0watchdog/20-21swapper/207:55:392
249940,0watchdog/20-21swapper/207:40:392
218859944,0cyclictest0-21swapper/211:46:062
218859944,0cyclictest0-21swapper/208:45:442
218859944,0cyclictest0-21swapper/208:45:442
218859943,0cyclictest6883-21awk08:15:392
218859943,0cyclictest3556-21lldpd08:51:482
218859943,0cyclictest3556-21lldpd08:03:482
218859943,0cyclictest3556-21lldpd06:55:452
218859943,0cyclictest3514-21snmpd07:26:412
218859943,0cyclictest3241-21irqbalance09:56:032
218859943,0cyclictest3241-21irqbalance09:21:232
218859943,0cyclictest3241-21irqbalance07:30:432
218859943,0cyclictest31700-21grep07:45:402
218859943,0cyclictest24007-21awk07:15:402
218859943,0cyclictest23053-21awk06:50:402
218859943,0cyclictest1131-21awk08:35:412
218839944,0cyclictest0-21swapper/110:06:331
218839944,0cyclictest0-21swapper/110:01:391
218839943,0cyclictest4224-21awk10:20:381
218839943,0cyclictest3943-21awk11:00:381
218839943,0cyclictest3556-21lldpd11:32:251
218839943,0cyclictest3514-21snmpd12:03:111
218839943,0cyclictest3514-21snmpd11:44:561
218839943,0cyclictest3514-21snmpd11:06:251
218839943,0cyclictest3514-21snmpd10:30:261
218839943,0cyclictest3514-21snmpd09:05:261
218839943,0cyclictest3514-21snmpd09:00:411
218839943,0cyclictest3514-21snmpd08:10:411
218839943,0cyclictest27861-21awk11:55:381
218839943,0cyclictest26826-21awk08:30:401
218839943,0cyclictest24965-21awk07:40:391
218839943,0cyclictest12826-21awk09:25:401
218829944,0cyclictest0-21swapper/009:46:300
218829943,0cyclictest8596-21kworker/0:109:08:160
218829943,0cyclictest5538-21awk09:20:400
218829943,0cyclictest4937-21awk07:25:410
218829943,0cyclictest3514-21snmpd11:45:410
218829943,0cyclictest3514-21snmpd11:21:250
218829943,0cyclictest32631-21awk08:10:380
218829943,0cyclictest12579-21awk07:55:390
179940,0migration/16342-21kthreadcore07:25:421
179940,0migration/15492-21kthreadcore09:40:411
179940,0migration/14532-21kthreadcore11:20:411
179940,0migration/14134-21irqcore07:00:411
179940,0migration/131520-21kthreadcore11:35:431
179940,0migration/130896-21kthreadcore10:35:421
179940,0migration/130430-21kthreadcore12:15:391
179940,0migration/129583-21idleruntime06:55:401
179940,0migration/128779-21kthreadcore08:05:411
179940,0migration/128037-21latency_hist08:55:251
179940,0migration/125475-21kthreadcore06:50:421
179940,0migration/125318-21latency09:30:411
179940,0migration/124924-21rt-features08:25:461
179940,0migration/123034-21kthreadcore11:50:401
179940,0migration/120049-21taskset10:10:351
179940,0migration/112394-21kthreadcore11:25:401
169940,0watchdog/1704-21cut09:35:431
169940,0watchdog/130547-21kthreadcore09:15:431
169940,0watchdog/125945-21ssh12:13:461
169940,0watchdog/124351-21kthreadcore09:50:431
169940,0watchdog/111439-21munin-node10:25:381
169940,0watchdog/10-21swapper/110:59:221
169940,0watchdog/10-21swapper/110:51:341
169940,0watchdog/10-21swapper/110:44:061
169940,0watchdog/10-21swapper/109:45:391
169940,0watchdog/10-21swapper/108:40:391
149940,0migration/08561-21kthreadcore09:40:420
149940,0migration/05262-21kthreadcore12:00:410
149940,0migration/032500-21diskmemload12:15:430
149940,0migration/028496-21nfs4_client07:15:430
149940,0migration/028368-21irqcore11:15:380
149940,0migration/026433-21runrttasks10:53:280
149940,0migration/025499-21sh12:12:290
149940,0migration/022107-21aten_r1power_cu06:50:360
149940,0migration/017820-21kthreadcore08:45:420
149940,0migration/017820-21kthreadcore08:45:420
149940,0migration/011947-21sh09:25:350
149940,0migration/011376-21fschecks_time07:30:380
29715030,0irq/26-eth0-rx-0-21swapper/308:45:363
29715030,0irq/26-eth0-rx-0-21swapper/308:45:363
29715030,0irq/26-eth0-rx-0-21swapper/308:05:353
29715030,0irq/26-eth0-rx-0-21swapper/307:40:353
29715030,0irq/26-eth0-rx-0-21swapper/307:30:363
29715030,0irq/26-eth0-rx-0-21swapper/307:20:373
29715030,0irq/26-eth0-rx-0-21swapper/307:15:263
29715030,0irq/26-eth0-rx-0-21swapper/307:13:353
29715030,0irq/26-eth0-rx-0-21swapper/307:00:363
29715030,0irq/26-eth0-rx-0-21swapper/306:52:523
218859932,0cyclictest3556-21lldpd08:31:482
218839933,0cyclictest0-21swapper/107:35:301
218839932,0cyclictest6853-21awk08:15:391
218839932,0cyclictest3514-21snmpd08:20:261
218839932,0cyclictest25003-21sensors11:10:441
218829933,0cyclictest8596-21kworker/0:110:05:280
218829933,0cyclictest8596-21kworker/0:110:00:420
218829933,0cyclictest8596-21kworker/0:109:15:260
218829933,0cyclictest8596-21kworker/0:107:40:320
218829933,0cyclictest8596-21kworker/0:107:35:380
218829933,0cyclictest8596-21kworker/0:107:10:280
218829933,0cyclictest11939-21kworker/0:011:40:290
218829933,0cyclictest11939-21kworker/0:011:25:270
169930,0watchdog/10-21swapper/107:55:391
169930,0watchdog/10-21swapper/107:15:391
29715020,0irq/26-eth0-rx-0-21swapper/308:25:363
29715020,0irq/26-eth0-rx-0-21swapper/307:50:363
218839921,0cyclictest18145-21latency_hist08:00:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional