You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-25 - 02:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sat Jul 25, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2109929190,0sleep20-21swapper/218:46:252
304450560,0irq/25-eth10-21swapper/318:46:393
2103823223,0sleep10-21swapper/118:45:441
2094422622,0sleep00-21swapper/018:44:390
213779980,0cyclictest0-21swapper/200:03:492
213759988,0cyclictest23172-21awk23:54:030
339970,0migration/322061-21sh23:32:023
339960,0migration/324972-21munin-node22:54:033
339960,0migration/320919-21sh00:12:263
339960,0migration/31558-21sh21:15:063
339960,0migration/313327-21sh00:08:323
29725060,0irq/27-eth0-tx-24519-21munin-node20:04:010
29725060,0irq/27-eth0-tx-10186-21munin-node19:29:060
29715060,0irq/26-eth0-rx-12122-21iostat_ios21:24:063
29715060,0irq/26-eth0-rx-0-21swapper/323:49:013
213779966,0cyclictest0-21swapper/223:24:052
213759966,0cyclictest11563-21sensors00:04:120
213759960,0cyclictest0-21swapper/021:49:110
149960,0migration/07408-21sh22:41:360
149960,0migration/025931-21sh21:31:370
149960,0migration/024269-21kthreadcore19:14:070
149960,0migration/02266-21sh21:17:160
149960,0migration/010174-21sh21:21:500
339950,0migration/34882-21taskset00:02:003
339950,0migration/331976-21idleruntime20:09:053
339950,0migration/330689-21latency_hist23:38:493
339950,0migration/328433-21munin-node23:34:093
339950,0migration/327440-21kthreadcore22:34:073
339950,0migration/32723-21unixbench_multi19:19:123
339950,0migration/327117-21kthreadcore19:39:073
339950,0migration/324556-21if_eth123:14:033
339950,0migration/321463-21munin-node19:09:113
339950,0migration/321458-21kthreadcore20:24:063
339950,0migration/32119-21munin-node21:59:033
339950,0migration/31870-21kthreadcore19:44:073
339950,0migration/31847-21kthreadcore22:39:053
339950,0migration/318057-21unixbench-2d20:19:133
339950,0migration/313309-21aten_r1power_fr20:44:003
339950,0migration/310940-21taskset19:53:433
339950,0migration/310589-21sh22:04:043
29725050,0irq/27-eth0-tx-30171-21munin-node19:44:110
29725050,0irq/27-eth0-tx-20904-21iostat_ios20:49:060
29725050,0irq/27-eth0-tx-0-21swapper/019:03:090
29725050,0irq/27-eth0-tx-0-21swapper/018:54:030
29715050,0irq/26-eth0-rx-9199-21pmu-power20:59:103
29715050,0irq/26-eth0-rx-5963-21sh22:59:183
29715050,0irq/26-eth0-rx-32470-21open_inodes21:34:103
29715050,0irq/26-eth0-rx-26257-21stty21:54:003
29715050,0irq/26-eth0-rx-20351-21kthreadcore22:09:073
29715050,0irq/26-eth0-rx-12473-21cut23:04:073
29715050,0irq/26-eth0-rx-0-21swapper/323:44:083
29715050,0irq/26-eth0-rx-0-21swapper/323:24:073
29715050,0irq/26-eth0-rx-0-21swapper/323:21:443
29715050,0irq/26-eth0-rx-0-21swapper/323:13:493
29715050,0irq/26-eth0-rx-0-21swapper/322:50:443
29715050,0irq/26-eth0-rx-0-21swapper/322:19:033
29715050,0irq/26-eth0-rx-0-21swapper/322:18:033
29715050,0irq/26-eth0-rx-0-21swapper/321:52:453
29715050,0irq/26-eth0-rx-0-21swapper/321:43:563
29715050,0irq/26-eth0-rx-0-21swapper/321:09:093
29715050,0irq/26-eth0-rx-0-21swapper/321:04:373
29715050,0irq/26-eth0-rx-0-21swapper/320:54:023
29715050,0irq/26-eth0-rx-0-21swapper/319:24:073
29715050,0irq/26-eth0-rx-0-21swapper/300:16:203
213779954,0cyclictest1759-21grep22:19:042
213779950,0cyclictest0-21swapper/222:30:012
213769955,0cyclictest0-21swapper/123:39:361
213769955,0cyclictest0-21swapper/122:15:151
213769955,0cyclictest0-21swapper/121:25:351
213769955,0cyclictest0-21swapper/100:11:451
213769950,0cyclictest0-21swapper/121:13:171
213759955,0cyclictest19648-21diskmemload22:27:010
213759955,0cyclictest0-21swapper/022:29:320
213759954,0cyclictest30365-21ps19:19:050
213759952,0cyclictest4352-21grep22:19:080
213759950,0cyclictest0-21swapper/023:23:270
213759950,0cyclictest0-21swapper/021:09:550
19150,0ktimersoftd/10-21swapper/123:59:281
19150,0ktimersoftd/10-21swapper/123:30:181
19150,0ktimersoftd/10-21swapper/100:17:451
179950,0migration/19454-21kthreadcore23:24:061
179950,0migration/120699-21kthreadcore20:24:061
169950,0watchdog/130647-21ssh22:34:121
169950,0watchdog/116682-21cut19:09:051
149950,0migration/08824-21irqcore23:04:050
149950,0migration/08408-21kthreadcore20:59:080
149950,0migration/06185-21munin-node20:14:040
149950,0migration/04148-21kthreadcore20:34:090
149950,0migration/032688-21kthreadcore23:59:080
149950,0migration/032261-21kthreadcore23:39:070
149950,0migration/031869-21proc_pri21:54:100
149950,0migration/031735-21kthreadcore21:34:080
149950,0migration/030888-21munin-node22:14:060
149950,0migration/029245-21kthreadcore22:54:070
149950,0migration/027788-21kthreadcore20:29:070
149950,0migration/02769-21kthreadcore20:09:080
149950,0migration/025376-21kthreadcore23:14:040
149950,0migration/022189-21fschecks_time18:49:040
149950,0migration/022077-21latency_hist23:53:490
149950,0migration/021548-21latency_hist00:13:490
149950,0migration/017745-21kthreadcore22:49:060
149950,0migration/017727-21kthreadcore19:09:070
149950,0migration/015563-21fschecks_time23:29:020
149950,0migration/014376-21fschecks_time00:09:030
149950,0migration/012338-21kthreadcore22:04:060
149950,0migration/011932-21kthreadcore21:44:060
149950,0migration/010776-21kthreadcore20:39:100
149950,0migration/010104-21taskset21:43:430
339940,0migration/36218-21kthreadcore21:19:063
339940,0migration/328739-21fschecks_count18:54:043
339940,0migration/327328-21switchtime18:49:133
339940,0migration/323879-21kthreadcore19:14:063
339940,0migration/319850-21if_err_eth121:29:053
339940,0migration/315976-21kthreadcore19:54:093
339940,0migration/310705-21kthreadcore19:04:083
29725040,0irq/27-eth0-tx-21765-21kthreadcore21:09:082
29725040,0irq/27-eth0-tx-17341-21ssh21:24:382
29725040,0irq/27-eth0-tx-16846-21telnet19:34:020
29725040,0irq/27-eth0-tx-0-21swapper/223:58:212
29725040,0irq/27-eth0-tx-0-21swapper/223:35:232
29725040,0irq/27-eth0-tx-0-21swapper/223:09:032
29725040,0irq/27-eth0-tx-0-21swapper/223:03:552
29725040,0irq/27-eth0-tx-0-21swapper/222:45:242
29725040,0irq/27-eth0-tx-0-21swapper/222:36:442
29725040,0irq/27-eth0-tx-0-21swapper/222:25:222
29725040,0irq/27-eth0-tx-0-21swapper/222:13:592
29725040,0irq/27-eth0-tx-0-21swapper/222:09:022
29725040,0irq/27-eth0-tx-0-21swapper/220:54:532
29715040,0irq/26-eth0-rx-0-21swapper/323:54:023
29715040,0irq/26-eth0-rx-0-21swapper/322:46:373
29715040,0irq/26-eth0-rx-0-21swapper/322:29:193
29715040,0irq/26-eth0-rx-0-21swapper/322:24:313
29715040,0irq/26-eth0-rx-0-21swapper/321:39:013
29715040,0irq/26-eth0-rx-0-21swapper/320:49:083
249940,0watchdog/23397-21kthreadcore23:19:082
249940,0watchdog/232261-21kthreadcore23:59:082
249940,0watchdog/216681-21sed19:09:052
249940,0watchdog/20-21swapper/223:39:082
249940,0watchdog/20-21swapper/219:19:052
213779943,0cyclictest7430-21awk23:44:062
213779943,0cyclictest7352-21awk20:39:072
213779943,0cyclictest4483-21awk20:59:052
213779943,0cyclictest3556-21lldpd21:35:372
213779943,0cyclictest3556-21lldpd20:14:382
213779943,0cyclictest3556-21lldpd00:15:382
213779943,0cyclictest3514-21snmpd23:31:122
213779943,0cyclictest3514-21snmpd22:41:272
213779943,0cyclictest3514-21snmpd21:59:422
213779943,0cyclictest3514-21snmpd21:38:572
213779943,0cyclictest3514-21snmpd21:30:422
213779943,0cyclictest3514-21snmpd20:44:122
213779943,0cyclictest3514-21snmpd20:27:112
213779943,0cyclictest3514-21snmpd19:49:112
213779943,0cyclictest3241-21irqbalance22:59:182
213779943,0cyclictest3241-21irqbalance22:54:182
213779943,0cyclictest3241-21irqbalance22:50:182
213779943,0cyclictest3241-21irqbalance22:05:082
213779943,0cyclictest3241-21irqbalance21:54:182
213779943,0cyclictest3241-21irqbalance21:44:082
213779943,0cyclictest3241-21irqbalance20:49:192
213779943,0cyclictest3241-21irqbalance20:34:492
213779943,0cyclictest3241-21irqbalance20:09:192
213779943,0cyclictest3241-21irqbalance20:07:292
213779943,0cyclictest3241-21irqbalance20:00:392
213779943,0cyclictest3241-21irqbalance19:55:392
213779943,0cyclictest3241-21irqbalance19:44:092
213779943,0cyclictest3241-21irqbalance19:38:592
213779943,0cyclictest3241-21irqbalance19:34:192
213779943,0cyclictest3241-21irqbalance19:29:592
213779943,0cyclictest3241-21irqbalance19:24:292
213779943,0cyclictest3241-21irqbalance19:15:092
213779943,0cyclictest3241-21irqbalance19:06:292
213779943,0cyclictest3241-21irqbalance18:59:292
213779943,0cyclictest3241-21irqbalance18:55:092
213779943,0cyclictest3241-21irqbalance00:09:082
213779943,0cyclictest24509-21awk23:14:022
213779943,0cyclictest22424-21awk18:49:052
213779943,0cyclictest15218-21awk23:49:052
213779943,0cyclictest13011-21awk20:19:042
213779940,0cyclictest18339-21munin-node21:49:032
213779940,0cyclictest0-21swapper/221:16:422
213769943,0cyclictest6243-21sadc20:38:501
213769943,0cyclictest3556-21lldpd23:37:081
213769943,0cyclictest3556-21lldpd22:19:071
213769943,0cyclictest3556-21lldpd20:59:371
213769943,0cyclictest3556-21lldpd19:49:071
213769943,0cyclictest3556-21lldpd19:33:071
213769943,0cyclictest3556-21lldpd00:04:381
213769943,0cyclictest3381-21awk21:39:051
213769943,0cyclictest25374-21awk20:04:031
213769943,0cyclictest23277-21awk23:54:041
213769943,0cyclictest22566-21awk18:49:061
213769940,0cyclictest0-21swapper/120:39:521
213759944,0cyclictest25475-21kworker/0:020:56:420
213759943,0cyclictest523-21awk22:59:040
213759943,0cyclictest3514-21snmpd22:09:120
213759943,0cyclictest3514-21snmpd21:05:570
213759943,0cyclictest25803-21awk22:34:050
213759943,0cyclictest24315-21awk19:39:040
213759943,0cyclictest14067-21sadc23:48:490
213759943,0cyclictest14064-21awk20:44:030
179940,0migration/19107-21kthreadcore20:14:071
179940,0migration/17555-21kthreadcore21:19:061
179940,0migration/1453-21kthreadcore18:54:081
179940,0migration/13792-21unixbench_singl19:44:121
179940,0migration/130431-21kthreadcore21:14:071
179940,0migration/129140-21kthreadcore20:54:071
179940,0migration/127241-21kthreadcore23:14:051
179940,0migration/126799-21kthreadcore19:14:071
179940,0migration/125517-21munin-node20:49:101
179940,0migration/123740-21timerwakeupswit19:59:121
179940,0migration/122667-21kthreadcore21:49:071
179940,0migration/120520-21kthreadcore19:34:081
179940,0migration/117465-21taskset19:57:141
179940,0migration/116232-21switchtime21:44:121
179940,0migration/11598-21irqrtprio22:39:051
179940,0migration/113894-21sshd23:04:181
179940,0migration/113230-21aten_r1power_cu20:44:001
179940,0migration/112767-21kthreadcore19:04:081
179940,0migration/110195-21fschecks_count22:04:021
169940,0watchdog/18187-21kthreadcore23:44:081
169940,0watchdog/16077-21ssh22:59:401
169940,0watchdog/13534-21iostat_ios18:59:051
169940,0watchdog/130475-21uname19:19:051
169940,0watchdog/10-21swapper/123:49:081
169940,0watchdog/10-21swapper/123:09:041
169940,0watchdog/10-21swapper/122:50:561
169940,0watchdog/10-21swapper/121:59:081
149940,0migration/029176-21runrttasks23:34:400
149940,0migration/017642-21kthreadcore23:09:070
149940,0migration/012890-21kthreadcore21:24:070
29725030,0irq/27-eth0-tx-0-21swapper/221:04:022
29715030,0irq/26-eth0-rx-0-21swapper/320:39:043
29715030,0irq/26-eth0-rx-0-21swapper/320:38:323
29715030,0irq/26-eth0-rx-0-21swapper/320:29:013
29715030,0irq/26-eth0-rx-0-21swapper/320:03:593
29715030,0irq/26-eth0-rx-0-21swapper/320:01:133
29715030,0irq/26-eth0-rx-0-21swapper/319:28:533
249930,0watchdog/20-21swapper/221:19:002
213779932,0cyclictest26485-21awk20:29:062
213769933,0cyclictest3556-21lldpd21:34:071
213769932,0cyclictest4390-21awk19:24:051
213769932,0cyclictest3556-21lldpd19:40:081
213769932,0cyclictest32044-21awk20:09:051
213769932,0cyclictest31141-21sendmail_mailst20:29:111
213769932,0cyclictest25505-21ssh21:30:201
213769932,0cyclictest24173-21cat22:53:491
213769932,0cyclictest15683-21kthreadcore20:19:061
213769932,0cyclictest12492-21awk21:04:051
213769930,0cyclictest771-21cut23:19:071
213759933,0cyclictest25475-21kworker/0:023:23:510
213759933,0cyclictest25475-21kworker/0:022:44:030
213759933,0cyclictest25475-21kworker/0:021:59:250
213759933,0cyclictest25475-21kworker/0:020:23:540
213759933,0cyclictest25475-21kworker/0:020:18:520
213759933,0cyclictest25475-21kworker/0:019:58:540
213759933,0cyclictest25475-21kworker/0:019:53:560
213759933,0cyclictest25475-21kworker/0:019:48:520
213759933,0cyclictest25475-21kworker/0:019:23:520
213759933,0cyclictest25475-21kworker/0:019:03:520
169930,0watchdog/122767-21sensors22:29:121
169930,0watchdog/117563-21ssh22:08:521
169930,0watchdog/10-21swapper/122:44:041
169930,0watchdog/10-21swapper/122:25:441
169930,0watchdog/10-21swapper/121:54:081
339920,0migration/35874-21diskstats20:14:033
29715020,0irq/26-eth0-rx-0-21swapper/319:34:023
29715020,0irq/26-eth0-rx-0-21swapper/318:58:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional