You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 11:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Mon Jun 29, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2807226059,0sleep20-21swapper/218:48:512
304450390,0irq/25-eth10-21swapper/318:51:063
2803423824,0sleep00-21swapper/018:48:280
2826922622,0sleep10-21swapper/118:51:101
35160,0ktimersoftd/30-21swapper/321:01:353
339960,0migration/39715-21sh21:17:523
339960,0migration/3645-21sh23:53:273
339960,0migration/332436-21munin-node22:51:393
339960,0migration/329551-21timerandwakeup19:36:423
339960,0migration/324102-21sh23:26:383
339960,0migration/317492-21sh00:05:083
339960,0migration/31407-21sh23:14:203
339960,0migration/310969-21df_abs23:01:303
339960,0migration/310230-21latency_hist23:41:193
29715060,0irq/26-eth0-rx-3154-21perl22:56:333
29715060,0irq/26-eth0-rx-19596-21perl21:06:333
29715060,0irq/26-eth0-rx-16653-21python22:21:403
283539960,0cyclictest26505-21diskmemload23:39:222
27160,0ktimersoftd/20-21swapper/221:25:402
149960,0migration/09112-21sh20:58:060
149960,0migration/0729-21sh22:32:020
149960,0migration/0680-21sh20:52:450
149960,0migration/018572-21sh21:25:100
149960,0migration/017806-21sh21:04:150
149960,0migration/017232-21sh22:22:180
149960,0migration/01210-21sh21:52:280
35150,0ktimersoftd/30-21swapper/322:29:433
339950,0migration/38741-21kthreadcore20:11:363
339950,0migration/36523-21kthreadcore00:16:373
339950,0migration/35744-21munin-node22:36:353
339950,0migration/35584-21kthreadcore23:56:363
339950,0migration/34259-21irqrtprio21:36:353
339950,0migration/33770-21munin-node21:56:343
339950,0migration/330513-21kthreadcore18:51:363
339950,0migration/330335-21kthreadcore00:11:363
339950,0migration/328655-21kthreadcore20:01:373
339950,0migration/327245-21munin-node21:11:313
339950,0migration/324580-21kthreadcore20:46:373
339950,0migration/319404-21df_abs22:06:323
339950,0migration/318215-21latency_hist23:46:193
339950,0migration/318215-21latency_hist23:46:193
339950,0migration/317967-21kthreadcore19:06:363
339950,0migration/314777-21kthreadcore22:01:363
339950,0migration/313138-21kthreadcore21:41:373
339950,0migration/312341-21irqcore21:21:363
29725050,0irq/27-eth0-tx-8331-21netstat22:16:402
29725050,0irq/27-eth0-tx-32411-21munin-node20:31:380
29725050,0irq/27-eth0-tx-2576-21munin-node22:36:312
29725050,0irq/27-eth0-tx-19634-21kernelversion23:46:342
29725050,0irq/27-eth0-tx-19634-21kernelversion23:46:342
29725050,0irq/27-eth0-tx-19109-21munin-node20:21:310
29725050,0irq/27-eth0-tx-19011-21munin-node22:06:332
29725050,0irq/27-eth0-tx-12436-21munin-node20:16:320
29725050,0irq/27-eth0-tx-11291-21cpuspeed_turbos22:01:312
29725050,0irq/27-eth0-tx-10025-21ssh23:20:402
29725050,0irq/27-eth0-tx-0-21swapper/223:31:502
29725050,0irq/27-eth0-tx-0-21swapper/222:34:532
29725050,0irq/27-eth0-tx-0-21swapper/019:55:410
29725050,0irq/27-eth0-tx-0-21swapper/019:00:320
29715050,0irq/26-eth0-rx-28819-21tr23:31:363
29715050,0irq/26-eth0-rx-2318-21ssh20:56:283
29715050,0irq/26-eth0-rx-0-21swapper/323:36:333
29715050,0irq/26-eth0-rx-0-21swapper/323:21:393
29715050,0irq/26-eth0-rx-0-21swapper/323:16:423
29715050,0irq/26-eth0-rx-0-21swapper/322:43:243
29715050,0irq/26-eth0-rx-0-21swapper/322:11:373
29715050,0irq/26-eth0-rx-0-21swapper/321:51:183
29715050,0irq/26-eth0-rx-0-21swapper/321:26:273
29715050,0irq/26-eth0-rx-0-21swapper/319:31:373
283539955,0cyclictest0-21swapper/223:22:012
283539955,0cyclictest0-21swapper/222:53:012
283539955,0cyclictest0-21swapper/221:18:062
283519950,0cyclictest0-21swapper/123:55:371
283519950,0cyclictest0-21swapper/122:40:341
283519950,0cyclictest0-21swapper/100:06:011
283509955,0cyclictest20894-21kworker/0:122:06:440
283509955,0cyclictest0-21swapper/022:46:340
283509954,0cyclictest26672-21ssh23:11:200
283509950,0cyclictest18450-21munin-run23:06:190
259950,0migration/230336-21idleruntime19:16:342
259950,0migration/223547-21sh22:26:362
259950,0migration/220683-21kthreadcore23:06:352
249950,0watchdog/20-21swapper/223:56:372
249950,0watchdog/20-21swapper/221:26:332
179950,0migration/19320-21taskset20:58:431
179950,0migration/132235-21idleruntime20:06:341
179950,0migration/130178-21kthreadcore22:51:361
179950,0migration/12800-21df_abs23:16:331
179950,0migration/127711-21kthreadcore20:01:361
179950,0migration/116123-21missed_timers22:41:381
179950,0migration/110727-21munin-node23:41:321
169950,0watchdog/116664-21unixbench-2d23:01:411
149950,0migration/08519-21needreboot21:36:380
149950,0migration/05965-21irqrtprio19:46:360
149950,0migration/05595-21kthreadcore22:36:350
149950,0migration/04346-21kthreadcore00:16:370
149950,0migration/031071-21kthreadcore21:11:370
149950,0migration/03028-21fschecks_count22:56:330
149950,0migration/02998-21gltestperf23:36:320
149950,0migration/028843-21kthreadcore23:31:360
149950,0migration/026964-21aten_r1power_po22:51:300
149950,0migration/026188-21taskset20:47:460
149950,0migration/026134-21kthreadcore20:01:360
149950,0migration/025603-21kthreadcore19:11:370
149950,0migration/023622-21missed_timers00:06:370
149950,0migration/022589-21taskset19:11:150
149950,0migration/020908-21kthreadcore23:46:350
149950,0migration/020908-21kthreadcore23:46:350
149950,0migration/020096-21idleruntime21:26:340
149950,0migration/019314-21kthreadcore19:31:370
149950,0migration/01821-21taskset22:14:350
149950,0migration/01778-21kthreadcore19:16:380
149950,0migration/017689-21sh00:05:430
149950,0migration/016228-21kthreadcore22:01:360
8215040,0irq/24-0000:00:10581-21cat22:41:192
8215040,0irq/24-0000:00:0-21swapper/220:21:192
339940,0migration/35570-21kthreadcore18:56:363
339940,0migration/33286-21kthreadcore20:31:373
339940,0migration/329407-21kthreadcore20:26:353
339940,0migration/32416-21kthreadcore19:41:373
339940,0migration/320659-21kthreadcore20:21:343
339940,0migration/319344-21fschecks_time23:06:323
29725040,0irq/27-eth0-tx-26191-21munin-node00:11:302
29725040,0irq/27-eth0-tx-16508-21ssh23:41:482
29725040,0irq/27-eth0-tx-10823-21munin-node22:21:322
29725040,0irq/27-eth0-tx-0-21swapper/223:26:332
29725040,0irq/27-eth0-tx-0-21swapper/223:01:292
29725040,0irq/27-eth0-tx-0-21swapper/222:46:492
29725040,0irq/27-eth0-tx-0-21swapper/222:11:322
29725040,0irq/27-eth0-tx-0-21swapper/221:51:242
29725040,0irq/27-eth0-tx-0-21swapper/221:31:382
29725040,0irq/27-eth0-tx-0-21swapper/200:16:222
29725040,0irq/27-eth0-tx-0-21swapper/019:41:400
29725040,0irq/27-eth0-tx-0-21swapper/019:36:300
29725040,0irq/27-eth0-tx-0-21swapper/019:01:340
29715040,0irq/26-eth0-rx-26505-21diskmemload00:06:343
29715040,0irq/26-eth0-rx-0-21swapper/322:47:553
29715040,0irq/26-eth0-rx-0-21swapper/322:31:243
29715040,0irq/26-eth0-rx-0-21swapper/322:17:203
29715040,0irq/26-eth0-rx-0-21swapper/321:51:423
29715040,0irq/26-eth0-rx-0-21swapper/321:31:203
29715040,0irq/26-eth0-rx-0-21swapper/320:51:323
29715040,0irq/26-eth0-rx-0-21swapper/320:39:513
283539944,0cyclictest0-21swapper/219:26:482
283539943,0cyclictest3556-21lldpd20:35:442
283539943,0cyclictest3556-21lldpd20:26:142
283539943,0cyclictest3556-21lldpd20:11:442
283539943,0cyclictest3556-21lldpd20:08:142
283539943,0cyclictest3556-21lldpd19:47:142
283539943,0cyclictest3556-21lldpd00:03:142
283539943,0cyclictest3514-21snmpd20:00:552
283539943,0cyclictest3241-21irqbalance21:39:372
283539943,0cyclictest3241-21irqbalance19:12:182
283539943,0cyclictest3241-21irqbalance18:57:082
283539943,0cyclictest3241-21irqbalance18:51:382
283539943,0cyclictest3218-21awk20:56:342
283539943,0cyclictest31381-21awk19:41:342
283539943,0cyclictest27749-21awk20:51:352
283539943,0cyclictest27595-21awk23:11:332
283539943,0cyclictest26661-21awk20:26:332
283539943,0cyclictest20126-21awk21:46:362
283539943,0cyclictest19123-21awk00:06:342
283539943,0cyclictest17019-21awk19:06:342
283539943,0cyclictest10380-21awk19:01:352
283539940,0cyclictest0-21swapper/219:35:182
283519943,0cyclictest3789-21awk21:56:341
283519943,0cyclictest3556-21lldpd20:19:141
283519943,0cyclictest3514-21snmpd23:39:271
283519943,0cyclictest3514-21snmpd22:48:261
283519943,0cyclictest3514-21snmpd19:51:101
283519943,0cyclictest3514-21snmpd19:11:241
283519943,0cyclictest3429-21awk22:56:351
283519943,0cyclictest3241-21irqbalance23:47:471
283519943,0cyclictest3241-21irqbalance23:47:471
283519943,0cyclictest3241-21irqbalance22:22:471
283519943,0cyclictest3241-21irqbalance22:02:171
283519943,0cyclictest30471-21awk19:16:351
283519943,0cyclictest26733-21awk20:26:341
283519943,0cyclictest19962-21awk22:06:351
283519943,0cyclictest18960-21awk19:56:341
283519943,0cyclictest14363-21awk20:41:351
283519943,0cyclictest12099-21awk21:21:341
283509944,0cyclictest0-21swapper/021:32:570
283509944,0cyclictest0-21swapper/021:20:580
283509944,0cyclictest0-21swapper/021:08:400
283509943,0cyclictest3514-21snmpd21:46:550
283509943,0cyclictest3514-21snmpd20:40:390
283509943,0cyclictest3241-21irqbalance22:18:270
283509943,0cyclictest3241-21irqbalance21:59:370
283509943,0cyclictest27261-21awk23:51:340
283509943,0cyclictest24112-21awk23:26:380
283509943,0cyclictest19615-21awk23:06:330
283509943,0cyclictest11692-21awk22:41:350
283509940,0cyclictest25754-21munin-node20:26:390
259940,0migration/226662-21kthreadcore20:01:362
259940,0migration/221465-21kthreadcore20:46:362
259940,0migration/219196-21cpuspeed_turbos21:06:322
259940,0migration/217404-21kthreadcore20:41:372
249940,0watchdog/229570-21wc19:36:422
249940,0watchdog/227701-21kthreadcore23:51:372
249940,0watchdog/20-21swapper/222:56:332
249940,0watchdog/20-21swapper/221:56:372
249940,0watchdog/20-21swapper/221:41:332
249940,0watchdog/20-21swapper/221:11:372
249940,0watchdog/20-21swapper/221:01:332
249940,0watchdog/20-21swapper/220:36:372
249940,0watchdog/20-21swapper/219:21:422
179940,0migration/18446-21latency21:16:371
179940,0migration/17082-21kthreadcore21:36:361
179940,0migration/15759-21aten_r1power_en20:11:301
179940,0migration/14618-21idleruntime19:21:331
179940,0migration/14143-21kernelversion18:56:351
179940,0migration/132664-21cpuspeed_turbos20:31:321
179940,0migration/132217-21missed_timers23:11:371
179940,0migration/132013-21munin-node00:11:401
179940,0migration/128077-21irqcore22:31:341
179940,0migration/127836-21fschecks_time21:51:321
179940,0migration/126613-21kthreadcore19:36:361
179940,0migration/125239-21taskset23:28:131
179940,0migration/121292-21kthreadcore22:26:351
179940,0migration/121288-21kthreadcore21:06:371
179940,0migration/118785-21kthreadcore19:31:361
179940,0migration/114580-21kthreadcore23:21:371
179940,0migration/113103-21kthreadcore19:26:361
179940,0migration/110731-21aten_r1power_fr21:01:321
169940,0watchdog/127838-21perl21:11:331
169940,0watchdog/124182-21tr20:46:371
169940,0watchdog/124087-21mdstat23:06:371
169940,0watchdog/119798-21fschecks_count21:26:331
169940,0watchdog/119164-21sh21:46:291
169940,0watchdog/11815-21python18:51:421
169940,0watchdog/110255-21gltestperf19:01:341
169940,0watchdog/10-21swapper/123:31:371
169940,0watchdog/10-21swapper/122:16:331
169940,0watchdog/10-21swapper/120:36:371
169940,0watchdog/10-21swapper/119:51:341
169940,0watchdog/10-21swapper/119:41:341
169940,0watchdog/10-21swapper/100:16:361
149940,0migration/09025-21kthreadcore20:11:360
149940,0migration/027266-21irqcore00:11:340
149940,0migration/015631-21kthreadcore19:26:370
35130,0ktimersoftd/30-21swapper/320:07:503
339930,0migration/39358-21runrttasks19:21:393
29725030,0irq/27-eth0-tx-0-21swapper/019:56:310
29715030,0irq/26-eth0-rx-0-21swapper/320:45:433
29715030,0irq/26-eth0-rx-0-21swapper/319:56:303
29715030,0irq/26-eth0-rx-0-21swapper/319:46:303
29715030,0irq/26-eth0-rx-0-21swapper/319:26:313
29715030,0irq/26-eth0-rx-0-21swapper/319:20:363
29715030,0irq/26-eth0-rx-0-21swapper/319:14:443
283519933,0cyclictest2938-21awk23:56:341
283519932,0cyclictest357-21grep20:51:431
283519932,0cyclictest3556-21lldpd20:22:441
283519932,0cyclictest3514-21snmpd00:06:271
283519932,0cyclictest22600-21sadc19:11:191
283519932,0cyclictest11325-21grep21:41:321
283509933,0cyclictest20894-21kworker/0:121:41:280
283509933,0cyclictest20894-21kworker/0:120:41:310
283509933,0cyclictest20894-21kworker/0:120:06:310
283509933,0cyclictest20894-21kworker/0:119:21:310
283509933,0cyclictest20894-21kworker/0:118:51:210
283509933,0cyclictest11614-21kworker/0:123:56:220
283509933,0cyclictest11614-21kworker/0:123:21:200
283509933,0cyclictest11614-21kworker/0:123:16:200
283509933,0cyclictest11614-21kworker/0:122:26:240
283509932,0cyclictest16106-21sensors23:41:410
249930,0watchdog/20-21swapper/219:51:342
169930,0watchdog/10-21swapper/122:11:251
169930,0watchdog/10-21swapper/121:32:451
29715020,0irq/26-eth0-rx-0-21swapper/320:16:463
29715020,0irq/26-eth0-rx-0-21swapper/319:51:233
29715020,0irq/26-eth0-rx-0-21swapper/319:01:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional