You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-05 - 02:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sun Apr 05, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1095023625,0sleep00-21swapper/018:59:040
304450350,0irq/25-eth10-21swapper/318:54:303
1065823525,0sleep20-21swapper/218:55:422
1083923222,0sleep10-21swapper/118:57:491
4160,0ktimersoftd/00-21swapper/023:03:220
339960,0migration/39900-21sh21:19:263
339960,0migration/37766-21sh23:19:203
339960,0migration/37618-21sh22:37:053
339960,0migration/332388-21sh21:51:143
339960,0migration/330964-21sh23:32:093
339960,0migration/329805-21munin-node20:49:403
339960,0migration/324820-21sh22:08:343
339960,0migration/315521-21sh22:41:433
29725060,0irq/27-eth0-tx-15303-21munin-node20:39:370
29725060,0irq/27-eth0-tx-10268-21fschecks_time20:59:370
29715060,0irq/26-eth0-rx-8319-21kthreadcore19:44:403
29715060,0irq/26-eth0-rx-25777-21diskstats22:09:353
29715060,0irq/26-eth0-rx-0-21swapper/323:00:593
19160,0ktimersoftd/10-21swapper/123:03:221
149960,0migration/0939-21sh21:33:030
149960,0migration/08285-21sh21:35:520
149960,0migration/032051-21munin-node23:34:340
149960,0migration/019341-21kthreadcore23:24:390
149960,0migration/014163-21pmu-power20:34:440
35150,0ktimersoftd/30-21swapper/321:40:033
339950,0migration/36837-21munin-node21:54:433
339950,0migration/35606-21kthreadcore20:54:403
339950,0migration/35604-21kthreadcore20:04:403
339950,0migration/331586-21runrttasks00:14:343
339950,0migration/331523-21missed_timers21:29:403
339950,0migration/325584-21munin-node19:09:383
339950,0migration/324277-21sh21:05:573
339950,0migration/324257-21runrttasks22:48:563
339950,0migration/321177-21munin-node19:54:413
339950,0migration/319281-21munin-node19:29:363
339950,0migration/318960-21irqrtprio21:44:383
339950,0migration/318044-21kthreadcore20:39:413
339950,0migration/316304-21munin-node23:44:373
339950,0migration/316063-21runrttasks22:22:133
339950,0migration/31202-21kthreadcore19:14:393
339950,0migration/311967-21kthreadcore23:19:403
29725050,0irq/27-eth0-tx-591-21munin-node22:34:452
29725050,0irq/27-eth0-tx-24607-21munin-node19:09:420
29725050,0irq/27-eth0-tx-23496-21munin-node21:04:452
29725050,0irq/27-eth0-tx-1863-21munin-node20:29:350
29725050,0irq/27-eth0-tx-0-21swapper/200:15:022
29715050,0irq/26-eth0-rx-7213-21ssh22:56:433
29715050,0irq/26-eth0-rx-5262-21kthreadcore22:14:393
29715050,0irq/26-eth0-rx-31807-21ssh22:31:333
29715050,0irq/26-eth0-rx-16463-21sh23:24:353
29715050,0irq/26-eth0-rx-0-21swapper/323:59:393
29715050,0irq/26-eth0-rx-0-21swapper/323:39:443
29715050,0irq/26-eth0-rx-0-21swapper/323:04:393
29715050,0irq/26-eth0-rx-0-21swapper/322:53:563
29715050,0irq/26-eth0-rx-0-21swapper/321:35:053
29715050,0irq/26-eth0-rx-0-21swapper/321:24:303
29715050,0irq/26-eth0-rx-0-21swapper/321:14:323
29715050,0irq/26-eth0-rx-0-21swapper/321:12:403
29715050,0irq/26-eth0-rx-0-21swapper/321:01:343
29715050,0irq/26-eth0-rx-0-21swapper/319:49:413
29715050,0irq/26-eth0-rx-0-21swapper/300:24:413
29715050,0irq/26-eth0-rx-0-21swapper/300:05:463
27150,0ktimersoftd/20-21swapper/221:40:032
249950,0watchdog/229020-21kthreadcore22:49:412
249950,0watchdog/222935-21cut19:54:422
249950,0watchdog/22121-21kthreadcore22:54:412
249950,0watchdog/210493-21kthreadcore23:39:412
249950,0watchdog/210242-21cut19:19:422
19150,0ktimersoftd/10-21swapper/121:40:031
179950,0migration/17835-21latency_hist23:19:231
179950,0migration/126263-21mdstat20:19:421
179950,0migration/125521-21idleruntime22:49:381
179950,0migration/118822-21irqcore21:44:381
149950,0migration/09666-21kthreadcore19:44:400
149950,0migration/09460-21aten_r1power_en21:59:330
149950,0migration/09286-21kthreadcore19:19:420
149950,0migration/07571-21fschecks_count00:19:370
149950,0migration/03782-21kthreadcore19:39:400
149950,0migration/027884-21cpuspeed_turbos20:24:350
149950,0migration/02588-21kthreadcore20:04:390
149950,0migration/025716-21kthreadcore00:09:410
149950,0migration/024646-21fschecks_time23:29:360
149950,0migration/023667-21taskset23:08:080
149950,0migration/021967-21kthreadcore19:54:410
149950,0migration/021422-21fschecks_count20:19:370
149950,0migration/021187-21munin-node00:04:440
149950,0migration/020849-21kthreadcore21:04:410
149950,0migration/018976-21kthreadcore22:04:390
149950,0migration/018077-21aten_r1power_po19:04:350
149950,0migration/017732-21kthreadcore19:49:420
149950,0migration/014578-21sh23:41:480
149950,0migration/014039-21timerwakeupswit23:19:450
149950,0migration/013995-21taskset20:13:220
149950,0migration/012068-21kthreadcore21:39:400
149950,0migration/011257-21kthreadcore23:59:390
110519950,0cyclictest0-21swapper/121:57:431
110509955,0cyclictest0-21swapper/023:12:540
110509955,0cyclictest0-21swapper/022:53:310
110509955,0cyclictest0-21swapper/021:16:430
110509955,0cyclictest0-21swapper/000:14:520
110509950,0cyclictest0-21swapper/022:44:130
110509950,0cyclictest0-21swapper/021:49:490
4140,0ktimersoftd/00-21swapper/023:17:480
339940,0migration/329493-21kthreadcore20:24:403
339940,0migration/327910-21kthreadcore19:34:393
339940,0migration/326301-21kthreadcore20:44:413
339940,0migration/324781-21kthreadcore20:19:413
339940,0migration/323911-21cpu23:49:343
339940,0migration/312171-21kthreadcore21:59:393
29725040,0irq/27-eth0-tx-9511-21aten2.4-expect21:59:332
29725040,0irq/27-eth0-tx-8413-21munin-node22:59:372
29725040,0irq/27-eth0-tx-6559-21runrttasks19:59:350
29725040,0irq/27-eth0-tx-31603-21ssh22:09:562
29725040,0irq/27-eth0-tx-23504-21ssh23:07:412
29725040,0irq/27-eth0-tx-23272-21ssh21:44:412
29725040,0irq/27-eth0-tx-18961-21munin-node21:24:372
29725040,0irq/27-eth0-tx-0-21swapper/223:54:312
29725040,0irq/27-eth0-tx-0-21swapper/223:49:482
29725040,0irq/27-eth0-tx-0-21swapper/223:45:002
29725040,0irq/27-eth0-tx-0-21swapper/223:35:492
29725040,0irq/27-eth0-tx-0-21swapper/223:30:122
29725040,0irq/27-eth0-tx-0-21swapper/223:24:332
29725040,0irq/27-eth0-tx-0-21swapper/223:10:112
29725040,0irq/27-eth0-tx-0-21swapper/222:45:282
29725040,0irq/27-eth0-tx-0-21swapper/222:41:302
29725040,0irq/27-eth0-tx-0-21swapper/222:16:162
29725040,0irq/27-eth0-tx-0-21swapper/222:05:112
29725040,0irq/27-eth0-tx-0-21swapper/221:54:522
29725040,0irq/27-eth0-tx-0-21swapper/221:19:352
29725040,0irq/27-eth0-tx-0-21swapper/200:05:032
29725040,0irq/27-eth0-tx-0-21swapper/020:54:400
29725040,0irq/27-eth0-tx-0-21swapper/018:59:440
29715040,0irq/26-eth0-rx-5200-21munin-node23:34:433
29715040,0irq/26-eth0-rx-0-21swapper/323:56:153
29715040,0irq/26-eth0-rx-0-21swapper/323:09:363
29715040,0irq/26-eth0-rx-0-21swapper/322:24:303
29715040,0irq/26-eth0-rx-0-21swapper/320:29:393
29715040,0irq/26-eth0-rx-0-21swapper/320:09:353
29715040,0irq/26-eth0-rx-0-21swapper/319:04:343
29715040,0irq/26-eth0-rx-0-21swapper/300:19:253
29715040,0irq/26-eth0-rx-0-21swapper/300:09:553
259940,0migration/29445-21kthreadcore00:19:402
259940,0migration/29159-21sh22:19:342
259940,0migration/27390-21sh23:59:242
259940,0migration/26420-21taskset23:15:262
259940,0migration/25279-21kthreadcore20:54:402
259940,0migration/227573-21if_eth019:59:382
259940,0migration/222598-21kthreadcore19:29:402
259940,0migration/216847-21kthreadcore20:39:402
249940,0watchdog/26559-21runrttasks21:34:292
249940,0watchdog/26559-21runrttasks19:09:462
249940,0watchdog/224633-21irqrtprio00:09:412
249940,0watchdog/22400-21hddtemp_smartct21:14:372
249940,0watchdog/217921-21ssh22:24:372
249940,0watchdog/213990-21perl19:49:382
249940,0watchdog/20-21swapper/220:14:422
179940,0migration/19683-21aten_r1power_cu20:59:351
179940,0migration/18664-21aten_r1power_cu20:34:351
179940,0migration/17986-21kthreadcore19:44:401
179940,0migration/17567-21latency_hist23:39:231
179940,0migration/130440-21timerwakeupswit19:09:451
179940,0migration/128733-21kthreadcore23:09:401
179940,0migration/127846-21kthreadcore23:49:401
179940,0migration/125902-21cpuspeed_turbos19:34:341
179940,0migration/124881-21sh21:26:451
179940,0migration/119029-21kthreadcore22:24:391
179940,0migration/118932-21kthreadcore23:44:401
179940,0migration/117920-21kthreadcore00:24:381
179940,0migration/116533-21fschecks_count23:24:351
179940,0migration/114412-21taskset00:23:101
179940,0migration/110897-21kthreadcore21:59:381
169940,0watchdog/19423-21diskmemload21:29:331
169940,0watchdog/18698-21munin-node22:39:411
169940,0watchdog/14458-21memory00:14:411
169940,0watchdog/13248-21cut22:34:411
169940,0watchdog/123770-21ssh00:09:371
169940,0watchdog/118476-21munin-node21:04:371
169940,0watchdog/116074-21sh20:39:371
169940,0watchdog/115430-21ssh00:04:291
169940,0watchdog/10-21swapper/121:49:331
169940,0watchdog/10-21swapper/120:14:381
169940,0watchdog/10-21swapper/119:19:421
149940,0migration/09315-21cpuspeed_turbos22:19:340
149940,0migration/030379-21kthreadcore20:49:410
149940,0migration/023513-21sh22:25:440
149940,0migration/010054-21aten_r1power_fr21:19:360
110569943,0cyclictest8592-21awk20:09:382
110569943,0cyclictest7532-21awk19:44:382
110569943,0cyclictest3556-21lldpd20:50:302
110569943,0cyclictest3514-21snmpd19:39:312
110569943,0cyclictest3241-21irqbalance23:20:512
110569943,0cyclictest3241-21irqbalance21:09:322
110569943,0cyclictest3241-21irqbalance19:36:022
110569943,0cyclictest3241-21irqbalance19:24:422
110569943,0cyclictest3241-21irqbalance18:59:322
110569943,0cyclictest32133-21awk19:14:372
110569943,0cyclictest28601-21awk20:24:382
110569943,0cyclictest2832-21awk20:29:362
110569943,0cyclictest27019-21awk21:29:372
110569943,0cyclictest26634-21awk21:49:382
110569943,0cyclictest26008-21awk22:29:372
110569943,0cyclictest21928-21awk20:19:392
110569943,0cyclictest18806-21awk19:04:382
110569943,0cyclictest1746-21awk20:04:372
110569943,0cyclictest15985-21awk00:24:362
110569943,0cyclictest10623-21awk20:59:392
110519944,0cyclictest0-21swapper/122:19:561
110519943,0cyclictest8543-21awk20:09:381
110519943,0cyclictest8327-21awk23:59:371
110519943,0cyclictest7248-21needreboot21:34:411
110519943,0cyclictest464-21awk23:34:381
110519943,0cyclictest3556-21lldpd23:57:021
110519943,0cyclictest3556-21lldpd23:23:011
110519943,0cyclictest3556-21lldpd22:44:321
110519943,0cyclictest3556-21lldpd22:15:001
110519943,0cyclictest3556-21lldpd21:14:301
110519943,0cyclictest3556-21lldpd21:10:001
110519943,0cyclictest3556-21lldpd20:48:301
110519943,0cyclictest3556-21lldpd19:39:311
110519943,0cyclictest3556-21lldpd19:14:291
110519943,0cyclictest3556-21lldpd19:04:281
110519943,0cyclictest28577-21awk20:24:381
110519943,0cyclictest26371-21awk22:09:381
110519943,0cyclictest24860-21awk23:29:371
110519943,0cyclictest18332-21awk22:04:381
110519943,0cyclictest17942-21kthreadcore19:49:421
110519943,0cyclictest17295-21awk23:04:361
110509944,0cyclictest0-21swapper/022:57:370
110509943,0cyclictest3556-21lldpd23:49:320
110509943,0cyclictest3556-21lldpd23:44:320
110509943,0cyclictest3514-21snmpd23:55:010
110509943,0cyclictest3514-21snmpd21:26:460
110509943,0cyclictest3241-21irqbalance22:09:510
110509943,0cyclictest26562-21awk19:34:370
110509943,0cyclictest25960-21awk22:29:370
110509943,0cyclictest1841-21awk22:14:360
110509943,0cyclictest17666-21awk22:44:370
110509943,0cyclictest15969-21awk00:24:360
110509943,0cyclictest15189-21awk20:14:390
110509943,0cyclictest13263-21awk19:24:380
35130,0ktimersoftd/30-21swapper/320:37:133
29715030,0irq/26-eth0-rx-0-21swapper/320:14:363
29715030,0irq/26-eth0-rx-0-21swapper/319:59:393
29715030,0irq/26-eth0-rx-0-21swapper/319:23:033
29715030,0irq/26-eth0-rx-0-21swapper/318:59:343
169930,0watchdog/19423-21diskmemload21:19:251
169930,0watchdog/10-21swapper/122:54:411
169930,0watchdog/10-21swapper/122:29:251
169930,0watchdog/10-21swapper/120:49:411
169930,0watchdog/10-21swapper/120:29:421
110569932,0cyclictest3514-21snmpd20:35:312
110569930,0cyclictest22979-21iostat_ios20:44:392
110519933,0cyclictest3556-21lldpd20:55:301
110519933,0cyclictest3556-21lldpd18:59:291
110519933,0cyclictest0-21swapper/120:01:311
110519932,0cyclictest20925-21awk19:54:401
110519930,0cyclictest0-21swapper/119:24:231
110509933,0cyclictest31109-21kworker/0:122:34:340
110509933,0cyclictest31109-21kworker/0:121:09:450
110509933,0cyclictest31109-21kworker/0:120:44:330
110509933,0cyclictest31109-21kworker/0:119:14:310
110509933,0cyclictest19810-21awk19:29:380
110509932,0cyclictest3514-21snmpd21:54:310
110509930,0cyclictest18754-40yum-updatesd-he21:44:380
29715020,0irq/26-eth0-rx-0-21swapper/319:40:043
29715020,0irq/26-eth0-rx-0-21swapper/319:24:463
169920,0watchdog/10-21swapper/120:04:261
169920,0watchdog/10-21swapper/119:29:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional