You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 05:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Wed Aug 05, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450970,0irq/25-eth10-21swapper/318:47:183
52524938,0sleep00-21swapper/018:44:220
46823531,0sleep20-21swapper/218:43:442
2692823425,0sleep10-21swapper/118:42:511
9359960,0cyclictest31483-21diskmemload22:12:302
9349966,0cyclictest31483-21diskmemload22:51:401
339960,0migration/36473-21sh20:51:333
339960,0migration/32814-21sh23:11:403
339960,0migration/319919-21sh22:19:143
29725060,0irq/27-eth0-tx-25006-21munin-node23:23:082
29715060,0irq/26-eth0-rx-15440-21cut22:38:053
29715060,0irq/26-eth0-rx-1081-21seq23:28:193
29715060,0irq/26-eth0-rx-0-21swapper/322:34:163
9359955,0cyclictest0-21swapper/223:17:092
9359955,0cyclictest0-21swapper/222:49:052
9359955,0cyclictest0-21swapper/221:21:162
9359950,0cyclictest0-21swapper/222:32:222
9359950,0cyclictest0-21swapper/221:39:522
9359950,0cyclictest0-21swapper/220:48:002
9359950,0cyclictest0-21swapper/200:07:172
9349955,0cyclictest0-21swapper/123:12:281
9349955,0cyclictest0-21swapper/121:47:011
9349955,0cyclictest0-21swapper/121:25:351
9349950,0cyclictest0-21swapper/123:44:471
9349950,0cyclictest0-21swapper/122:03:531
9349950,0cyclictest0-21swapper/120:58:371
9339950,0cyclictest25731-21idleruntime-cro00:07:460
8215050,0irq/24-0000:00:3020-21cat19:37:482
8215050,0irq/24-0000:00:3020-21cat19:37:482
8215050,0irq/24-0000:00:23145-21cat21:22:472
8215050,0irq/24-0000:00:22943-21munin-run19:52:482
8215050,0irq/24-0000:00:10597-21perf20:07:482
35150,0ktimersoftd/30-21swapper/321:44:223
339950,0migration/37369-21aten_r1power_vo20:53:003
339950,0migration/37143-21kthreadcore23:53:043
339950,0migration/36951-21munin-node18:48:103
339950,0migration/36251-21gltestperf22:13:023
339950,0migration/35911-21sh21:09:373
339950,0migration/34264-21irqcore19:38:043
339950,0migration/34264-21irqcore19:38:043
339950,0migration/33578-21kthreadcore21:28:033
339950,0migration/33118-21munin-node22:08:073
339950,0migration/330780-21aten_r1power_en20:22:563
339950,0migration/329490-21cpuspeed_turbos22:27:593
339950,0migration/329113-21timerwakeupswit21:23:083
339950,0migration/32862-21munin-node19:58:053
339950,0migration/327793-21latency_hist19:07:483
339950,0migration/327430-21kthreadcore20:18:073
339950,0migration/325946-21kthreadcore22:23:053
339950,0migration/324160-21timerandwakeup20:38:113
339950,0migration/323761-21taskset20:16:403
339950,0migration/32200-21munin-node00:13:043
339950,0migration/320828-21aten_r1power_fr22:43:003
339950,0migration/32032-21kthreadcore21:48:053
339950,0migration/319963-21fschecks_time23:23:013
339950,0migration/317572-21kthreadcore21:58:083
339950,0migration/315620-21aten2.4-expect19:22:593
339950,0migration/313896-21kthreadcore22:58:033
339950,0migration/313702-21kthreadcore23:38:043
339950,0migration/310932-21gltestperf23:58:013
339950,0migration/310557-21taskset20:07:313
29725050,0irq/27-eth0-tx-3542-21lldpd21:16:332
29725050,0irq/27-eth0-tx-31625-21sed21:48:042
29725050,0irq/27-eth0-tx-31095-21munin-node21:27:592
29725050,0irq/27-eth0-tx-15518-21grep22:38:052
29725050,0irq/27-eth0-tx-0-21swapper/222:58:042
29715050,0irq/26-eth0-rx-9507-21kthreadcore21:53:043
29715050,0irq/26-eth0-rx-4838-21kernelversion23:13:063
29715050,0irq/26-eth0-rx-31349-21cut23:48:043
29715050,0irq/26-eth0-rx-28617-21sh22:04:493
29715050,0irq/26-eth0-rx-24776-21sed23:43:133
29715050,0irq/26-eth0-rx-24345-21kthreadcore19:03:063
29715050,0irq/26-eth0-rx-16484-21sensors20:08:103
29715050,0irq/26-eth0-rx-15523-21runrttasks21:18:003
29715050,0irq/26-eth0-rx-11907-21fschecks_count23:18:013
29715050,0irq/26-eth0-rx-10334-21kthreadcore21:33:053
29715050,0irq/26-eth0-rx-0-21swapper/321:14:053
29715050,0irq/26-eth0-rx-0-21swapper/300:10:493
27150,0ktimersoftd/20-21swapper/223:32:452
259950,0migration/22790-21latency_hist23:32:472
259950,0migration/226519-21smart_sdb22:43:112
259950,0migration/222837-21needreboot20:13:062
259950,0migration/218344-21latency_hist20:37:472
259950,0migration/215258-21irqcore21:58:062
259950,0migration/212505-21sh21:53:472
19150,0ktimersoftd/10-21swapper/100:08:451
179950,0migration/18766-21switchtime23:33:091
179950,0migration/13295-21timerwakeupswit19:58:081
179950,0migration/130650-21latency_hist20:22:471
179950,0migration/127062-21kthreadcore19:53:051
179950,0migration/12302-21sh23:32:011
179950,0migration/122862-21kthreadcore22:43:051
179950,0migration/118888-21kthreadcore20:13:041
179950,0migration/116384-21kthreadcore21:58:071
179950,0migration/115081-21kthreadcore22:38:051
179950,0migration/114887-21munin-node18:58:021
179950,0migration/111915-21sh22:35:561
179950,0migration/11028-21kthreadcore22:28:051
169950,0watchdog/120283-21open_inodes21:38:091
169950,0watchdog/115285-21cut20:08:061
169950,0watchdog/113628-21kthreadcore23:58:051
9359943,0cyclictest9734-21awk19:18:052
9359943,0cyclictest3556-21lldpd23:03:042
9359943,0cyclictest3556-21lldpd21:33:032
9359943,0cyclictest3556-21lldpd21:08:032
9359943,0cyclictest31598-21awk20:23:002
9359943,0cyclictest28728-21awk19:08:032
9359943,0cyclictest24454-21awk21:03:002
9359943,0cyclictest15541-21awk18:58:052
9359943,0cyclictest10730-21awk19:43:022
9349943,0cyclictest3556-21lldpd19:18:041
9349943,0cyclictest3514-21snmpd23:48:071
9349943,0cyclictest3514-21snmpd22:12:521
9349943,0cyclictest3514-21snmpd21:32:521
9349943,0cyclictest3514-21snmpd21:12:521
9349943,0cyclictest3514-21snmpd21:08:071
9349943,0cyclictest3514-21snmpd00:16:371
9349943,0cyclictest3241-21irqbalance22:59:251
9349943,0cyclictest3241-21irqbalance20:53:161
9349943,0cyclictest3241-21irqbalance20:33:161
9349943,0cyclictest3157-21awk23:53:021
9349943,0cyclictest23040-21awk19:28:031
9349943,0cyclictest17497-21awk19:48:041
9349940,0cyclictest1452-21munin-node19:08:111
9339944,0cyclictest0-21swapper/023:51:460
9339943,0cyclictest5997-21awk20:28:050
9339943,0cyclictest5035-21awk20:03:030
9339943,0cyclictest3556-21lldpd19:19:040
9339943,0cyclictest3514-21snmpd23:28:070
9339943,0cyclictest3514-21snmpd23:19:370
9339943,0cyclictest3514-21snmpd23:14:070
9339943,0cyclictest3514-21snmpd23:09:070
9339943,0cyclictest3514-21snmpd21:35:220
9339943,0cyclictest3514-21snmpd19:32:220
9339943,0cyclictest3241-21irqbalance23:58:150
9339943,0cyclictest3241-21irqbalance23:24:050
9339943,0cyclictest3241-21irqbalance22:38:250
9339943,0cyclictest3241-21irqbalance22:09:250
9339943,0cyclictest3241-21irqbalance21:48:550
9339943,0cyclictest3241-21irqbalance21:43:150
9339943,0cyclictest3241-21irqbalance21:42:050
9339943,0cyclictest3241-21irqbalance21:08:060
9339943,0cyclictest3241-21irqbalance19:52:260
9339943,0cyclictest3241-21irqbalance19:37:260
9339943,0cyclictest3241-21irqbalance19:04:560
9339943,0cyclictest3241-21irqbalance00:10:050
9339943,0cyclictest32050-21awk21:28:010
9339943,0cyclictest28696-21awk19:08:020
9339943,0cyclictest2437-21awk00:13:050
9339943,0cyclictest22921-21awk22:03:030
9339943,0cyclictest22269-21awk22:23:030
9339943,0cyclictest19221-21awk20:38:030
9339943,0cyclictest11728-21awk23:38:010
9339940,0cyclictest21019-21idleruntime-cro19:02:480
339940,0migration/37931-21aten_r1power_cu18:53:003
339940,0migration/36844-21kthreadcore20:28:063
339940,0migration/34330-21kthreadcore19:13:043
339940,0migration/331986-21kthreadcore22:48:053
339940,0migration/329539-21runrttasks19:57:203
339940,0migration/325329-21cpuspeed_turbos20:43:013
339940,0migration/32532-21taskset19:34:173
339940,0migration/318403-21kthreadcore20:58:053
29725040,0irq/27-eth0-tx-6559-21runrttasks23:48:512
29725040,0irq/27-eth0-tx-10674-21ssh22:54:162
29725040,0irq/27-eth0-tx-0-21swapper/223:52:542
29725040,0irq/27-eth0-tx-0-21swapper/223:42:562
29725040,0irq/27-eth0-tx-0-21swapper/223:38:112
29725040,0irq/27-eth0-tx-0-21swapper/223:07:512
29725040,0irq/27-eth0-tx-0-21swapper/222:33:012
29725040,0irq/27-eth0-tx-0-21swapper/222:18:232
29725040,0irq/27-eth0-tx-0-21swapper/222:03:012
29725040,0irq/27-eth0-tx-0-21swapper/220:58:322
29725040,0irq/27-eth0-tx-0-21swapper/200:12:522
29725040,0irq/27-eth0-tx-0-21swapper/020:48:050
29725040,0irq/27-eth0-tx-0-21swapper/020:43:040
29725040,0irq/27-eth0-tx-0-21swapper/020:22:570
29725040,0irq/27-eth0-tx-0-21swapper/020:18:010
29725040,0irq/27-eth0-tx-0-21swapper/019:58:020
29725040,0irq/27-eth0-tx-0-21swapper/019:53:070
29725040,0irq/27-eth0-tx-0-21swapper/019:13:020
29725040,0irq/27-eth0-tx-0-21swapper/018:48:040
29715040,0irq/26-eth0-rx-10353-21sh22:53:173
29715040,0irq/26-eth0-rx-0-21swapper/323:32:583
29715040,0irq/26-eth0-rx-0-21swapper/323:02:583
29715040,0irq/26-eth0-rx-0-21swapper/321:38:153
29715040,0irq/26-eth0-rx-0-21swapper/321:03:053
29715040,0irq/26-eth0-rx-0-21swapper/300:03:043
259940,0migration/28453-21irqrtprio20:53:042
259940,0migration/27429-21runrttasks18:49:492
259940,0migration/26613-21irqcore22:13:042
259940,0migration/229720-21kthreadcore00:08:052
259940,0migration/226419-21kthreadcore20:18:072
259940,0migration/224334-21sshd19:53:032
259940,0migration/221362-21munin-node19:03:002
259940,0migration/215638-21kthreadcore20:33:082
259940,0migration/21480-21kthreadcore19:58:032
259940,0migration/211787-21cpuspeed_turbos23:18:002
249940,0watchdog/25331-21cpuspeed_turbos20:28:022
249940,0watchdog/2226022chrt21:42:532
249940,0watchdog/221992-21ssh22:23:012
249940,0watchdog/221208-21python19:23:102
249940,0watchdog/210760-21perl23:58:012
249940,0watchdog/210433-21kthreadcore18:53:062
179940,0migration/19039-21kthreadcore21:53:041
179940,0migration/18378-21fschecks_count18:53:021
179940,0migration/1677-21kthreadcore22:08:041
179940,0migration/16377-21kthreadcore19:38:051
179940,0migration/16377-21kthreadcore19:38:051
179940,0migration/131826-21irqcore20:23:011
179940,0migration/12939-21kthreadcore21:48:051
179940,0migration/124175-21kthreadcore19:03:061
179940,0migration/12163-21irqcore18:48:041
179940,0migration/120517-21irqcore23:23:031
179940,0migration/119840-21kthreadcore00:03:041
179940,0migration/11967-21kthreadcore21:28:031
179940,0migration/115521-21latency_hist19:22:481
179940,0migration/114955-21kthreadcore22:18:041
179940,0migration/113837-21kthreadcore23:38:041
179940,0migration/113829-21kthreadcore23:18:041
179940,0migration/110583-21taskset23:15:551
169940,0watchdog/13243-21kthreadcore20:48:051
169940,0watchdog/12896-21munin-node19:13:021
169940,0watchdog/126344-21ssh23:03:531
169940,0watchdog/121597-21ssh21:18:411
169940,0watchdog/10-21swapper/122:53:171
169940,0watchdog/10-21swapper/121:04:131
169940,0watchdog/10-21swapper/120:43:051
169940,0watchdog/10-21swapper/120:28:021
149940,0migration/09260-21sh23:33:470
149940,0migration/08163-21munin-node21:13:030
149940,0migration/07585-21df_abs20:53:010
149940,0migration/04888-21sh22:32:590
149940,0migration/032566-21kthreadcore22:28:040
149940,0migration/028885-21sh21:03:030
149940,0migration/027765-21kthreadcore21:23:030
149940,0migration/020318-21sh22:20:280
149940,0migration/020200-21kthreadcore20:13:050
149940,0migration/016260-21kthreadcore22:58:040
149940,0migration/016121-21munin-node21:18:030
149940,0migration/014335-21kthreadcore20:08:050
9359933,0cyclictest3556-21lldpd19:37:042
9359932,0cyclictest3556-21lldpd20:43:032
9359932,0cyclictest3031-21awk19:13:032
9349932,0cyclictest3241-21irqbalance20:37:561
9349932,0cyclictest29551-21grep19:33:011
9349930,0cyclictest5058-21if_eth120:03:031
9339932,0cyclictest8796-21awk18:53:040
9339932,0cyclictest3514-21snmpd22:52:520
9339932,0cyclictest3514-21snmpd22:42:520
9339932,0cyclictest3514-21snmpd19:22:520
9339932,0cyclictest3241-21irqbalance22:12:550
9339932,0cyclictest2844-21fschecks_time23:53:000
9339932,0cyclictest19543-21head23:43:030
9339932,0cyclictest12007-21expr20:33:020
9339932,0cyclictest10745-21awk19:43:030
9339930,0cyclictest6517-21diskstats21:52:590
9339930,0cyclictest29032-21diskstats22:48:000
9339930,0cyclictest15689-21grep21:58:070
9339930,0cyclictest0-21swapper/019:37:560
9339930,0cyclictest0-21swapper/019:37:560
339930,0migration/314691-21aten_r1power_vo18:58:013
29715030,0irq/26-eth0-rx-0-21swapper/319:42:593
29715030,0irq/26-eth0-rx-0-21swapper/319:28:063
29715030,0irq/26-eth0-rx-0-21swapper/319:18:443
249930,0watchdog/222841-21fschecks_time19:28:022
249930,0watchdog/20-21swapper/220:08:022
169930,0watchdog/10-21swapper/122:23:051
9339921,0cyclictest153512chrt20:57:580
9339920,0cyclictest0-21swapper/023:02:470
29715020,0irq/26-eth0-rx-0-21swapper/320:32:563
29715020,0irq/26-eth0-rx-0-21swapper/319:47:553
169920,0watchdog/10-21swapper/119:42:501
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional