You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 02:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Jun 14, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3121625040,0sleep20-21swapper/218:49:242
304450460,0irq/25-eth10-21swapper/318:48:343
3125523228,0sleep00-21swapper/018:49:540
3111723223,0sleep10-21swapper/118:48:151
315739976,0cyclictest1902-21memory20:28:030
339960,0migration/33894-21sh21:53:413
339960,0migration/329600-21sh21:11:533
339960,0migration/320048-21sh21:43:143
339960,0migration/312213-21sh21:59:243
315759960,0cyclictest0-21swapper/223:33:002
315759960,0cyclictest0-21swapper/221:08:352
29725060,0irq/27-eth0-tx-10010-21munin-node20:37:560
29715060,0irq/26-eth0-rx-19297-21python22:23:083
29715060,0irq/26-eth0-rx-16193-21cut22:43:023
259960,0migration/230828-21irqcore22:13:012
19160,0ktimersoftd/10-21swapper/123:16:051
8215050,0irq/24-0000:00:5377-21cat22:17:452
339950,0migration/37517-21kthreadcore20:08:043
339950,0migration/33222-21taskset23:34:373
339950,0migration/331780-21kthreadcore22:13:023
339950,0migration/327462-21taskset19:58:553
339950,0migration/326427-21switchtime00:08:073
339950,0migration/325324-21kthreadcore23:48:033
339950,0migration/321946-21cpuspeed_turbos22:07:573
339950,0migration/318430-21kthreadcore20:43:023
339950,0migration/317208-21kthreadcore19:03:033
339950,0migration/316631-21if_eth1.220:18:013
339950,0migration/31640-21kthreadcore23:13:023
339950,0migration/315553-21munin-node19:53:003
339950,0migration/311506-21kthreadcore19:23:033
315759950,0cyclictest0-21swapper/222:41:082
315759950,0cyclictest0-21swapper/220:59:172
315749955,0cyclictest0-21swapper/123:00:101
315749955,0cyclictest0-21swapper/122:25:101
315749955,0cyclictest0-21swapper/121:41:291
315749955,0cyclictest0-21swapper/121:37:331
315749955,0cyclictest0-21swapper/100:07:101
315749950,0cyclictest0-21swapper/120:53:151
315739955,0cyclictest0-21swapper/021:59:100
315739950,0cyclictest0-21swapper/023:37:520
315739950,0cyclictest0-21swapper/022:16:030
29725050,0irq/27-eth0-tx-3104-21latency21:33:042
29725050,0irq/27-eth0-tx-30861-21if_eth1.221:52:582
29725050,0irq/27-eth0-tx-29644-21grep23:12:582
29725050,0irq/27-eth0-tx-29308-21ssh22:12:162
29725050,0irq/27-eth0-tx-28317-21ssh23:52:152
29725050,0irq/27-eth0-tx-27132-21ssh00:09:192
29725050,0irq/27-eth0-tx-23728-21grep22:48:012
29725050,0irq/27-eth0-tx-14730-21munin-node19:52:590
29725050,0irq/27-eth0-tx-12302-21munin-run00:02:442
29725050,0irq/27-eth0-tx-0-21swapper/223:43:122
29725050,0irq/27-eth0-tx-0-21swapper/223:40:432
29725050,0irq/27-eth0-tx-0-21swapper/223:26:562
29725050,0irq/27-eth0-tx-0-21swapper/222:45:032
29725050,0irq/27-eth0-tx-0-21swapper/222:01:212
29725050,0irq/27-eth0-tx-0-21swapper/221:03:062
29725050,0irq/27-eth0-tx-0-21swapper/020:53:590
29725050,0irq/27-eth0-tx-0-21swapper/020:02:450
29725050,0irq/27-eth0-tx-0-21swapper/019:02:580
29715050,0irq/26-eth0-rx-9920-21kthreadcore21:18:013
29715050,0irq/26-eth0-rx-9920-21kthreadcore21:18:013
29715050,0irq/26-eth0-rx-6559-21runrttasks21:38:013
29715050,0irq/26-eth0-rx-5895-21hwlatdetect22:57:593
29715050,0irq/26-eth0-rx-4611-21ssh23:37:523
29715050,0irq/26-eth0-rx-40672sleep321:33:333
29715050,0irq/26-eth0-rx-32676-21cut20:03:043
29715050,0irq/26-eth0-rx-30521-21if_eth1.222:33:023
29715050,0irq/26-eth0-rx-28508-21ssh22:30:333
29715050,0irq/26-eth0-rx-27624-21ssh23:29:413
29715050,0irq/26-eth0-rx-27448-21awk22:48:073
29715050,0irq/26-eth0-rx-27409-21df19:37:583
29715050,0irq/26-eth0-rx-26390-21kthreadcore23:08:033
29715050,0irq/26-eth0-rx-19217-21unixbench_singl23:03:083
29715050,0irq/26-eth0-rx-16753-21kthreadcore21:03:023
29715050,0irq/26-eth0-rx-13415-21fschecks_time23:42:573
29715050,0irq/26-eth0-rx-0-21swapper/323:57:563
29715050,0irq/26-eth0-rx-0-21swapper/323:55:363
29715050,0irq/26-eth0-rx-0-21swapper/323:22:563
29715050,0irq/26-eth0-rx-0-21swapper/322:56:043
29715050,0irq/26-eth0-rx-0-21swapper/320:58:183
29715050,0irq/26-eth0-rx-0-21swapper/300:21:213
29715050,0irq/26-eth0-rx-0-21swapper/300:13:383
29715050,0irq/26-eth0-rx-0-21swapper/300:05:363
27150,0ktimersoftd/20-21swapper/200:03:352
259950,0migration/2428-21kthreadcore21:13:012
259950,0migration/231539-21kthreadcore00:13:012
259950,0migration/226868-21timerwakeupswit23:28:082
259950,0migration/224957-21kthreadcore20:23:032
259950,0migration/219435-21sendmail_mailqu22:03:072
249950,0watchdog/26023-21kthreadcore20:08:042
249950,0watchdog/21268-21kthreadcore20:53:042
179950,0migration/18620-21taskset20:11:061
179950,0migration/13501-21sh22:33:281
179950,0migration/131393-21kthreadcore19:38:031
179950,0migration/125389-21kthreadcore22:28:021
179950,0migration/123410-21kthreadcore00:08:011
179950,0migration/122554-21irqcore22:48:001
179950,0migration/122449-21gltestperf22:07:581
179950,0migration/122212-21kthreadcore19:33:041
179950,0migration/113186-21taskset21:20:561
179950,0migration/113186-21taskset21:20:561
179950,0migration/113018-21aten_r1power_po23:22:551
169950,0watchdog/19018-21aten_r1power_cu20:12:561
169950,0watchdog/129808-21df_inode22:52:551
149950,0migration/0761-21kthreadcore23:53:060
149950,0migration/04476-21sh21:55:210
149950,0migration/029135-21irqcore20:03:020
149950,0migration/028106-21sh00:12:080
149950,0migration/028001-21munin-node20:23:080
149950,0migration/027125-21sh23:28:170
149950,0migration/02653-21kthreadcore21:13:020
149950,0migration/024582-21irqrtprio20:48:030
149950,0migration/023587-21kthreadcore22:48:010
149950,0migration/021056-21sh22:06:400
149950,0migration/021055-21kthreadcore20:43:030
149950,0migration/020973-21sh21:05:510
149950,0migration/02047-21munin-node00:13:080
149950,0migration/019631-21sendmail_mailst21:43:080
149950,0migration/019206-21taskset00:04:330
149950,0migration/018818-21missed_timers22:43:030
149950,0migration/016608-21kthreadcore23:43:020
149950,0migration/016031-21kthreadcore19:28:030
149950,0migration/015027-21if_eth021:23:000
149950,0migration/013907-21munin-node23:02:590
149950,0migration/012424-21sh20:59:510
149950,0migration/011627-21runrttasks00:21:260
149950,0migration/010958-21switchtime22:58:080
149950,0migration/010814-21kthreadcore18:58:030
8215040,0irq/24-0000:00:0-21swapper/221:42:462
35140,0ktimersoftd/30-21swapper/320:40:343
339940,0migration/39977-21if_eth1.320:13:003
339940,0migration/39595-21taskset20:35:423
339940,0migration/37293-21kthreadcore23:18:033
339940,0migration/36030-21gltestperf22:37:573
339940,0migration/33496-21kthreadcore19:43:003
339940,0migration/324756-21kthreadcore21:48:023
339940,0migration/322077-21kthreadcore19:08:023
339940,0migration/320392-21latency_hist19:32:463
315759943,0cyclictest9858-21awk20:12:592
315759943,0cyclictest4205-21awk20:33:012
315759943,0cyclictest3556-21lldpd20:39:262
315759943,0cyclictest3556-21lldpd20:27:562
315759943,0cyclictest3556-21lldpd19:39:252
315759943,0cyclictest3556-21lldpd19:24:562
315759943,0cyclictest3556-21lldpd19:01:252
315759943,0cyclictest3514-21snmpd20:00:062
315759943,0cyclictest3514-21snmpd19:17:062
315759943,0cyclictest30275-21awk22:52:572
315759943,0cyclictest15710-21awk19:53:012
315759943,0cyclictest14594-21awk19:28:002
315759943,0cyclictest14117-21awk23:03:002
315759943,0cyclictest13684-21awk19:03:002
315749944,0cyclictest0-21swapper/121:24:181
315749944,0cyclictest0-21swapper/119:48:171
315749943,0cyclictest3556-21lldpd18:57:561
315749943,0cyclictest3514-21snmpd20:31:371
315749943,0cyclictest3514-21snmpd18:54:361
315749943,0cyclictest23221-21awk20:23:011
315749943,0cyclictest14051-21awk23:03:001
315749940,0cyclictest0-21swapper/120:38:161
315739944,0cyclictest28010-21kworker/0:223:23:060
315739944,0cyclictest28010-21kworker/0:218:52:590
315739944,0cyclictest0-21swapper/023:59:280
315739944,0cyclictest0-21swapper/021:49:520
315739943,0cyclictest8934-21awk19:48:000
315739943,0cyclictest3556-21lldpd22:41:280
315739943,0cyclictest3514-21snmpd22:55:070
315739943,0cyclictest3514-21snmpd19:38:060
315739943,0cyclictest3134-21awk20:08:010
315739943,0cyclictest29870-21sadc21:32:460
315739943,0cyclictest22596-21awk22:07:590
315739940,0cyclictest32761-21munin-node19:17:580
315739940,0cyclictest19449-21munin-node19:08:010
315739940,0cyclictest15684-21munin-node20:18:030
29725040,0irq/27-eth0-tx-0-21swapper/223:53:032
29725040,0irq/27-eth0-tx-0-21swapper/223:07:462
29725040,0irq/27-eth0-tx-0-21swapper/222:57:492
29725040,0irq/27-eth0-tx-0-21swapper/222:27:522
29725040,0irq/27-eth0-tx-0-21swapper/222:22:492
29725040,0irq/27-eth0-tx-0-21swapper/221:47:552
29725040,0irq/27-eth0-tx-0-21swapper/221:37:502
29725040,0irq/27-eth0-tx-0-21swapper/221:27:542
29725040,0irq/27-eth0-tx-0-21swapper/200:18:062
29725040,0irq/27-eth0-tx-0-21swapper/020:32:580
29715040,0irq/26-eth0-rx-0-21swapper/322:18:233
29715040,0irq/26-eth0-rx-0-21swapper/322:02:533
29715040,0irq/26-eth0-rx-0-21swapper/321:30:183
29715040,0irq/26-eth0-rx-0-21swapper/321:23:493
29715040,0irq/26-eth0-rx-0-21swapper/321:12:493
29715040,0irq/26-eth0-rx-0-21swapper/320:53:023
29715040,0irq/26-eth0-rx-0-21swapper/320:28:013
259940,0migration/26674-21fschecks_time21:17:572
259940,0migration/26674-21fschecks_time21:17:572
259940,0migration/24112-21kthreadcore19:18:032
259940,0migration/232286-21kthreadcore22:33:042
259940,0migration/224474-21kthreadcore19:33:052
259940,0migration/220414-21kthreadcore20:18:032
259940,0migration/217192-21fschecks_count20:42:592
259940,0migration/21266-21kthreadcore18:53:022
249940,0watchdog/26223-21kthreadcore23:18:032
249940,0watchdog/232121-21cut20:03:042
249940,0watchdog/219596-21netstat21:23:032
249940,0watchdog/20-21swapper/219:08:002
179940,0migration/19137-21kthreadcore00:18:031
179940,0migration/17688-21taskset19:45:481
179940,0migration/1656-21kthreadcore23:53:061
179940,0migration/15323-21fschecks_time23:37:581
179940,0migration/15322-21kthreadcore19:18:041
179940,0migration/13795-21fschecks_count20:33:001
179940,0migration/132177-21kthreadcore00:13:011
179940,0migration/122221-21aten_r1power_po21:27:561
179940,0migration/121951-21munin-node23:28:011
179940,0migration/121851-21fschecks_time23:07:581
179940,0migration/118397-21kthreadcore21:03:021
179940,0migration/116713-21taskset20:42:571
179940,0migration/111019-21kthreadcore19:23:021
179940,0migration/110028-21kthreadcore23:18:041
169940,0watchdog/17063-21kthreadcore20:58:041
169940,0watchdog/14779-21ssh21:15:071
169940,0watchdog/14008-21ssh22:14:231
169940,0watchdog/128611-21perl20:03:001
169940,0watchdog/126445-21kthreadcore20:48:041
169940,0watchdog/123900-21kthreadcore23:48:031
169940,0watchdog/114208-21ssh22:42:591
169940,0watchdog/113620-21if_eth023:42:591
169940,0watchdog/110321-21python23:58:071
169940,0watchdog/10-21swapper/121:52:591
169940,0watchdog/10-21swapper/121:43:031
169940,0watchdog/10-21swapper/119:53:081
169940,0watchdog/10-21swapper/119:03:001
149940,0migration/09797-21kthreadcore21:18:010
149940,0migration/09797-21kthreadcore21:18:010
149940,0migration/07592-21fschecks_count19:22:580
149940,0migration/023329-21diskmemload21:08:010
149940,0migration/022068-21irqrtprio23:48:020
149940,0migration/01383-21aten_r1power_en19:42:550
149940,0migration/01301-21kthreadcore23:13:020
4130,0ktimersoftd/028940-21sh23:32:480
4130,0ktimersoftd/012900-21runrttasks22:21:450
35130,0ktimersoftd/30-21swapper/319:30:243
315759933,0cyclictest2496-21tr19:43:002
315749933,0cyclictest0-21swapper/123:32:481
315749932,0cyclictest3556-21lldpd22:19:581
315749932,0cyclictest3556-21lldpd19:07:551
315749932,0cyclictest22910-21awk21:48:001
315749932,0cyclictest16566-21awk20:18:001
315739933,0cyclictest28010-21kworker/0:221:38:010
315739932,0cyclictest3556-21lldpd21:34:570
315739932,0cyclictest31126-21kthreadcore19:13:040
315739932,0cyclictest30469-21awk22:33:020
315739932,0cyclictest22441-21awk22:27:590
315739932,0cyclictest22098-21awk23:08:000
315739932,0cyclictest21302-21awk19:33:020
315739932,0cyclictest14283-21wc20:13:030
315739932,0cyclictest10656-21sensors23:18:080
29715030,0irq/26-eth0-rx-0-21swapper/320:47:583
29715030,0irq/26-eth0-rx-0-21swapper/319:52:443
29715030,0irq/26-eth0-rx-0-21swapper/319:13:033
29715030,0irq/26-eth0-rx-0-21swapper/318:52:563
249930,0watchdog/223359-21munin-node20:48:042
249930,0watchdog/20-21swapper/219:47:482
179930,0migration/111790-21ksmtuned22:39:091
169930,0watchdog/129788-21diskmemload21:57:471
169930,0watchdog/122243-21if_err_eth119:58:001
169930,0watchdog/117296-21kthreadcore22:03:031
169930,0watchdog/10-21swapper/121:07:561
169930,0watchdog/10-21swapper/119:28:041
169930,0watchdog/10-21swapper/119:13:281
315739920,0cyclictest0-21swapper/022:22:450
29715020,0irq/26-eth0-rx-0-21swapper/320:23:223
29715020,0irq/26-eth0-rx-0-21swapper/319:17:513
29715020,0irq/26-eth0-rx-0-21swapper/318:57:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional