You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 10:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Jan 09, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351780,0ktimersoftd/31709-21perf19:07:193
801323524,0sleep00-21swapper/019:03:070
830823425,0sleep20-21swapper/219:06:292
809023424,0sleep10-21swapper/119:04:001
84669965,0cyclictest8243-21sed21:07:432
84669960,0cyclictest0-21swapper/222:50:222
84659960,0cyclictest0-21swapper/121:32:431
35160,0ktimersoftd/30-21swapper/322:55:593
35160,0ktimersoftd/30-21swapper/300:32:573
339960,0migration/35506-21sh23:22:053
339960,0migration/34763-21sh23:38:033
339960,0migration/323846-21sh21:34:163
339960,0migration/322466-21sh23:50:433
339960,0migration/322037-21switchtime19:37:503
339960,0migration/320690-21sh23:28:313
339960,0migration/314108-21sh21:10:333
339960,0migration/312225-21fschecks_time22:27:423
29715060,0irq/26-eth0-rx-24653-21cut19:17:453
29715060,0irq/26-eth0-rx-20353-21cut21:52:453
29715060,0irq/26-eth0-rx-17081-21grep21:12:453
149960,0migration/030150-21kthreadcore22:17:450
149960,0migration/027928-21sh23:13:070
149960,0migration/016430-21sh21:31:490
84669950,0cyclictest0-21swapper/223:57:232
84669950,0cyclictest0-21swapper/223:34:212
84669950,0cyclictest0-21swapper/222:41:232
84669950,0cyclictest0-21swapper/221:40:512
84659955,0cyclictest0-21swapper/121:56:031
84659955,0cyclictest0-21swapper/121:50:131
84659955,0cyclictest0-21swapper/121:16:001
84659955,0cyclictest0-21swapper/120:07:531
84659955,0cyclictest0-21swapper/100:35:261
84659950,0cyclictest0-21swapper/123:12:211
84659950,0cyclictest0-21swapper/122:53:541
84659950,0cyclictest0-21swapper/122:23:291
84659950,0cyclictest0-21swapper/121:37:451
84659950,0cyclictest0-21swapper/100:27:241
84649955,0cyclictest0-21swapper/023:05:510
84649955,0cyclictest0-21swapper/022:03:060
84649955,0cyclictest0-21swapper/022:03:060
84649950,0cyclictest0-21swapper/023:46:310
84649950,0cyclictest0-21swapper/023:19:150
84649950,0cyclictest0-21swapper/023:09:470
8215050,0irq/24-0000:00:11386-21ssh23:03:112
339950,0migration/39629-21kthreadcore20:17:453
339950,0migration/39540-21munin-node21:27:443
339950,0migration/3897-21kthreadcore19:22:463
339950,0migration/38739-21kthreadcore23:42:443
339950,0migration/36900-21latency_hist21:07:293
339950,0migration/3515-21taskset21:41:103
339950,0migration/34305-21aten_r1power_vo22:42:413
339950,0migration/33814-21munin-node20:12:493
339950,0migration/327914-21turbostat22:37:283
339950,0migration/327777-21kthreadcore00:12:463
339950,0migration/327073-21gltestperf21:57:413
339950,0migration/32147-21latency_hist22:02:293
339950,0migration/32147-21latency_hist22:02:293
339950,0migration/319197-21munin-node20:27:433
339950,0migration/319122-21kthreadcore00:07:463
339950,0migration/317080-21aten_r1power_po00:27:403
339950,0migration/313641-21cpuspeed_turbos23:07:393
339950,0migration/311879-21munin-node23:22:513
339950,0migration/310905-21smart_sdb23:02:523
339950,0migration/310420-21kthreadcore19:07:443
29725050,0irq/27-eth0-tx-26331-21munin-node21:57:512
29725050,0irq/27-eth0-tx-18345-21munin-node20:27:412
29725050,0irq/27-eth0-tx-16404-21diskstats00:07:422
29725050,0irq/27-eth0-tx-15373-21munin-node19:12:482
29725050,0irq/27-eth0-tx-0-21swapper/223:42:132
29725050,0irq/27-eth0-tx-0-21swapper/222:35:082
29725050,0irq/27-eth0-tx-0-21swapper/221:53:012
29715050,0irq/26-eth0-rx-8935-21irqrtprio00:02:433
29715050,0irq/26-eth0-rx-6559-21runrttasks00:20:473
29715050,0irq/26-eth0-rx-31289-21kthreadcore22:37:453
29715050,0irq/26-eth0-rx-23452-21chrt20:03:413
29715050,0irq/26-eth0-rx-15273-21ssh00:24:023
29715050,0irq/26-eth0-rx-1327-21ssh21:42:313
29715050,0irq/26-eth0-rx-0-21swapper/323:52:493
29715050,0irq/26-eth0-rx-0-21swapper/323:36:483
29715050,0irq/26-eth0-rx-0-21swapper/323:12:393
29715050,0irq/26-eth0-rx-0-21swapper/322:49:173
29715050,0irq/26-eth0-rx-0-21swapper/322:23:383
29715050,0irq/26-eth0-rx-0-21swapper/322:12:443
29715050,0irq/26-eth0-rx-0-21swapper/321:50:353
29715050,0irq/26-eth0-rx-0-21swapper/321:26:043
29715050,0irq/26-eth0-rx-0-21swapper/321:17:333
29715050,0irq/26-eth0-rx-0-21swapper/319:27:443
259950,0migration/28554-21sh22:03:042
259950,0migration/28554-21sh22:03:042
259950,0migration/27081-21pmu-power21:42:492
259950,0migration/25542-21sh23:57:532
259950,0migration/227142-21latency_hist22:17:292
259950,0migration/225811-21kthreadcore20:07:442
259950,0migration/222886-21kthreadcore22:52:432
259950,0migration/220780-21sh23:28:452
259950,0migration/219633-21kthreadcore19:37:452
259950,0migration/218381-21kthreadcore21:12:452
259950,0migration/21589-21kthreadcore21:02:452
259950,0migration/214710-21kthreadcore19:57:442
249950,0watchdog/220433-21ssh23:10:022
179950,0migration/16178-21kthreadcore19:52:441
179950,0migration/132337-21gltestperf20:37:421
179950,0migration/124988-21sh20:52:501
179950,0migration/124049-21sh00:30:141
179950,0migration/118862-21kthreadcore00:07:461
179950,0migration/118227-21munin-node20:02:421
169950,0watchdog/17826-21runrttasks00:02:381
169950,0watchdog/110554-21missed_timers23:02:461
149950,0migration/09519-21kthreadcore20:17:450
149950,0migration/08779-21kthreadcore20:42:460
149950,0migration/08573-21sh00:22:300
149950,0migration/08391-21latency_hist00:22:280
149950,0migration/07873-21munin-node00:02:390
149950,0migration/05670-21runrttasks21:22:480
149950,0migration/05205-21kthreadcore19:27:440
149950,0migration/03259-21sh22:58:540
149950,0migration/032426-21runrttasks21:39:470
149950,0migration/032192-21fschecks_count23:57:410
149950,0migration/028828-21kthreadcore21:57:450
149950,0migration/027597-21kthreadcore19:42:460
149950,0migration/027563-21kthreadcore20:07:450
149950,0migration/023427-21sh21:17:160
149950,0migration/019615-21idleruntime-cro22:32:290
149950,0migration/018834-21kthreadcore19:37:450
149950,0migration/017890-21idleruntime21:32:430
149950,0migration/017608-21kthreadcore00:07:460
149950,0migration/016953-21sh00:27:370
149950,0migration/01425-21kthreadcore20:37:450
149950,0migration/012715-21sh23:24:190
149950,0migration/012504-21aten_r1power_en22:47:390
84669943,0cyclictest4890-21awk19:52:432
84669943,0cyclictest3556-21lldpd20:51:402
84669943,0cyclictest29528-21awk19:22:432
84669943,0cyclictest18226-21awk20:02:422
84669943,0cyclictest18114-21sadc20:27:292
84669940,0cyclictest0-21swapper/219:27:432
84659944,0cyclictest0-21swapper/122:18:261
84659943,0cyclictest5931-21awk20:17:431
84659943,0cyclictest3556-21lldpd23:47:441
84659943,0cyclictest3556-21lldpd23:13:151
84659943,0cyclictest3556-21lldpd20:46:101
84659943,0cyclictest3556-21lldpd20:12:411
84659943,0cyclictest3556-21lldpd19:32:411
84659943,0cyclictest3514-21snmpd19:13:421
84659943,0cyclictest3241-21irqbalance22:04:571
84659943,0cyclictest3241-21irqbalance22:04:571
84659943,0cyclictest26845-21awk20:57:431
84659943,0cyclictest17208-21awk19:37:431
84659943,0cyclictest12545-21awk20:22:421
84659943,0cyclictest12504-21awk22:27:441
84659943,0cyclictest11481-21awk22:07:421
84649943,0cyclictest3556-21lldpd22:37:450
84649943,0cyclictest3556-21lldpd21:21:110
84649943,0cyclictest3514-21snmpd21:02:270
84649943,0cyclictest31778-21awk23:37:440
84649943,0cyclictest25336-21awk00:12:440
84649943,0cyclictest24444-21awk23:52:430
84649943,0cyclictest22261-21kworker/0:023:34:340
84649943,0cyclictest22261-21kworker/0:022:09:560
84649943,0cyclictest22261-21kworker/0:021:42:580
84649943,0cyclictest21642-21awk22:52:410
84649943,0cyclictest19227-21awk20:27:430
339940,0migration/329244-21taskset19:45:343
339940,0migration/327627-21df_abs22:17:403
339940,0migration/310115-21taskset19:53:473
29725040,0irq/27-eth0-tx-7787-21telnet00:02:382
29725040,0irq/27-eth0-tx-29711-21mdstat00:12:472
29725040,0irq/27-eth0-tx-29651-21munin-node19:47:512
29725040,0irq/27-eth0-tx-26026-21ssh00:32:412
29725040,0irq/27-eth0-tx-18332-21ssh22:29:202
29725040,0irq/27-eth0-tx-11927-21ssh22:07:442
29725040,0irq/27-eth0-tx-0-21swapper/223:48:042
29725040,0irq/27-eth0-tx-0-21swapper/223:42:492
29725040,0irq/27-eth0-tx-0-21swapper/223:14:022
29725040,0irq/27-eth0-tx-0-21swapper/222:42:372
29725040,0irq/27-eth0-tx-0-21swapper/222:23:112
29725040,0irq/27-eth0-tx-0-21swapper/222:12:572
29725040,0irq/27-eth0-tx-0-21swapper/221:47:382
29725040,0irq/27-eth0-tx-0-21swapper/221:28:562
29725040,0irq/27-eth0-tx-0-21swapper/221:23:442
29725040,0irq/27-eth0-tx-0-21swapper/220:32:392
29725040,0irq/27-eth0-tx-0-21swapper/220:21:112
29725040,0irq/27-eth0-tx-0-21swapper/200:27:432
29725040,0irq/27-eth0-tx-0-21swapper/200:22:582
29725040,0irq/27-eth0-tx-0-21swapper/200:17:292
29715040,0irq/26-eth0-rx-27287-21kthreadcore20:57:453
29715040,0irq/26-eth0-rx-0-21swapper/323:58:263
29715040,0irq/26-eth0-rx-0-21swapper/322:57:303
29715040,0irq/26-eth0-rx-0-21swapper/322:08:313
29715040,0irq/26-eth0-rx-0-21swapper/320:52:443
29715040,0irq/26-eth0-rx-0-21swapper/320:42:413
259940,0migration/24765-21idleruntime-cro20:17:292
259940,0migration/227015-21kthreadcore19:17:452
259940,0migration/224652-21kthreadcore19:42:452
259940,0migration/223931-21aten_r1power_fr21:17:402
259940,0migration/218658-21kthreadcore21:32:462
259940,0migration/212842-21kthreadcore19:32:452
259940,0migration/211182-21kthreadcore23:22:462
249940,0watchdog/211108-21tr20:42:472
249940,0watchdog/20-21swapper/223:17:422
249940,0watchdog/20-21swapper/222:57:422
249940,0watchdog/20-21swapper/220:57:512
179940,0migration/17215-21aten_r1power_vo21:07:391
179940,0migration/16468-21kthreadcore22:42:461
179940,0migration/131428-21latency_hist20:37:291
179940,0migration/130968-21kthreadcore19:47:461
179940,0migration/129292-21ksmtuned19:46:071
179940,0migration/128096-21latency_hist22:37:281
179940,0migration/126715-21kthreadcore19:17:451
179940,0migration/126121-21kthreadcore23:52:451
179940,0migration/125612-21kthreadcore23:32:451
179940,0migration/121405-21kthreadcore22:32:441
179940,0migration/120080-21irqcore22:12:451
179940,0migration/11657-21kthreadcore23:57:451
179940,0migration/115404-21irqcore23:27:441
179940,0migration/113690-21sh23:44:351
179940,0migration/110913-21kthreadcore19:07:451
169940,0watchdog/18050-21rm21:44:031
169940,0watchdog/13853-21ssh23:00:141
169940,0watchdog/13026-21kthreadcore23:37:461
169940,0watchdog/120384-21ssh22:52:151
169940,0watchdog/112286-21ssh23:23:141
169940,0watchdog/10-21swapper/121:22:471
149940,0migration/030124-21fschecks_count19:47:420
149940,0migration/029646-21kthreadcore20:32:460
149940,0migration/029500-21kthreadcore00:32:450
149940,0migration/023657-21kthreadcore22:32:450
149940,0migration/020748-21kthreadcore21:52:450
149940,0migration/020163-21kthreadcore19:12:470
149940,0migration/019836-21fschecks_count20:52:410
149940,0migration/019680-21sh23:27:460
149940,0migration/015330-21kthreadcore19:57:440
149940,0migration/013872-21kthreadcore21:47:470
84669932,0cyclictest31646-21aten2.4-expect20:37:392
84669930,0cyclictest9073-21diskstats19:07:412
84659932,0cyclictest8704-21sensors19:27:491
84659932,0cyclictest3556-21lldpd21:17:421
84659932,0cyclictest3556-21lldpd21:05:101
84659932,0cyclictest3556-21lldpd19:22:411
84659932,0cyclictest3556-21lldpd00:12:441
84659932,0cyclictest18321-21sensors20:47:511
84649933,0cyclictest3556-21lldpd22:22:450
84649933,0cyclictest22261-21kworker/0:023:47:430
84649933,0cyclictest22261-21kworker/0:022:42:360
84649933,0cyclictest22261-21kworker/0:022:14:040
84649933,0cyclictest22261-21kworker/0:021:07:480
84649933,0cyclictest22261-21kworker/0:020:47:360
84649933,0cyclictest22261-21kworker/0:020:22:320
84649933,0cyclictest22261-21kworker/0:020:13:000
84649933,0cyclictest22261-21kworker/0:019:52:360
84649933,0cyclictest22261-21kworker/0:019:32:320
84649933,0cyclictest22261-21kworker/0:019:22:390
84649933,0cyclictest22261-21kworker/0:019:17:330
84649933,0cyclictest22261-21kworker/0:019:07:550
84649932,0cyclictest3514-21snmpd20:02:340
84649930,0cyclictest835-21perl21:02:420
339930,0migration/312838-21irqcore20:22:443
29715030,0irq/26-eth0-rx-0-21swapper/320:47:483
29715030,0irq/26-eth0-rx-0-21swapper/320:40:243
29715030,0irq/26-eth0-rx-0-21swapper/320:32:393
29715030,0irq/26-eth0-rx-0-21swapper/320:07:403
29715030,0irq/26-eth0-rx-0-21swapper/319:57:383
29715030,0irq/26-eth0-rx-0-21swapper/319:47:413
29715030,0irq/26-eth0-rx-0-21swapper/319:32:403
29715030,0irq/26-eth0-rx-0-21swapper/319:17:073
259930,0migration/225486-21runrttasks20:54:352
169930,0watchdog/18584-21ssh21:27:391
169930,0watchdog/10-21swapper/123:17:421
169930,0watchdog/10-21swapper/121:57:431
169930,0watchdog/10-21swapper/120:27:471
169930,0watchdog/10-21swapper/119:57:471
169930,0watchdog/10-21swapper/100:18:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional