You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 07:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Mar 29, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2109023827,0sleep20-21swapper/218:57:152
304450370,0irq/25-eth10-21swapper/318:57:053
2102123623,0sleep00-21swapper/018:56:250
2107322925,0sleep10-21swapper/118:57:021
214059999,0cyclictest0-21swapper/221:00:432
339960,0migration/38869-21sh23:33:183
339960,0migration/329444-21kthreadcore20:40:193
339960,0migration/32922-21sh21:49:373
339960,0migration/32842-21munin-node23:10:183
339960,0migration/325597-21sh22:21:333
339960,0migration/319441-21sh21:39:573
339960,0migration/311069-21sh21:54:483
29725060,0irq/27-eth0-tx-5427-21munin-node20:25:150
29725060,0irq/27-eth0-tx-24522-21munin-node20:15:180
29715060,0irq/26-eth0-rx-27263-21cut00:05:193
29715060,0irq/26-eth0-rx-25348-21sh22:41:483
214049960,0cyclictest0-21swapper/122:15:161
149960,0migration/019307-21sh21:39:340
149960,0migration/016971-21sh22:56:400
8215050,0irq/24-0000:00:12847-21munin-run20:55:022
339950,0migration/39207-21munin-node19:35:233
339950,0migration/38214-21needreboot22:30:203
339950,0migration/36098-21fschecks_time20:25:163
339950,0migration/330347-21aten_r1power_vo19:55:143
339950,0migration/329746-21kthreadcore23:05:193
339950,0migration/329089-21irqrtprio21:05:193
339950,0migration/328740-21taskset19:26:593
339950,0migration/328740-21taskset19:26:593
339950,0migration/325091-21aten_r1power_fr00:25:143
339950,0migration/323840-21kthreadcore22:00:203
339950,0migration/323820-21kthreadcore21:40:203
339950,0migration/32362-21irqcore23:50:183
339950,0migration/319890-21aten_r1power_po21:00:153
339950,0migration/317517-21sh22:58:163
29725050,0irq/27-eth0-tx-25430-21munin-node20:40:240
29725050,0irq/27-eth0-tx-24126-21load22:40:202
29725050,0irq/27-eth0-tx-2257-21ssh21:07:212
29725050,0irq/27-eth0-tx-19942-21if_err_eth1.222:20:162
29725050,0irq/27-eth0-tx-11753-21munin-node21:35:222
29725050,0irq/27-eth0-tx-0-21swapper/222:46:492
29725050,0irq/27-eth0-tx-0-21swapper/222:33:342
29725050,0irq/27-eth0-tx-0-21swapper/200:18:412
29725050,0irq/27-eth0-tx-0-21swapper/020:39:500
29725050,0irq/27-eth0-tx-0-21swapper/019:00:180
29715050,0irq/26-eth0-rx-8190-21kthreadcore22:10:193
29715050,0irq/26-eth0-rx-7529-21kthreadcore19:10:203
29715050,0irq/26-eth0-rx-360-21munin-node22:25:233
29715050,0irq/26-eth0-rx-31898-21tail20:20:153
29715050,0irq/26-eth0-rx-31576-21timerandwakeup23:45:263
29715050,0irq/26-eth0-rx-23479-21memory23:20:203
29715050,0irq/26-eth0-rx-1843-21ssh22:07:343
29715050,0irq/26-eth0-rx-16330-21taskset19:41:353
29715050,0irq/26-eth0-rx-15393-21kthreadcore21:15:193
29715050,0irq/26-eth0-rx-12272-21kthreadcore00:15:193
29715050,0irq/26-eth0-rx-0-21swapper/323:57:373
29715050,0irq/26-eth0-rx-0-21swapper/323:40:183
29715050,0irq/26-eth0-rx-0-21swapper/323:25:143
29715050,0irq/26-eth0-rx-0-21swapper/323:04:413
29715050,0irq/26-eth0-rx-0-21swapper/322:52:533
29715050,0irq/26-eth0-rx-0-21swapper/322:16:403
29715050,0irq/26-eth0-rx-0-21swapper/321:55:193
29715050,0irq/26-eth0-rx-0-21swapper/321:31:233
29715050,0irq/26-eth0-rx-0-21swapper/321:27:273
29715050,0irq/26-eth0-rx-0-21swapper/321:21:073
29715050,0irq/26-eth0-rx-0-21swapper/320:35:203
29715050,0irq/26-eth0-rx-0-21swapper/319:20:193
29715050,0irq/26-eth0-rx-0-21swapper/300:23:433
29715050,0irq/26-eth0-rx-0-21swapper/300:04:443
259950,0migration/25552-21timerwakeupswit20:45:262
259950,0migration/232428-21sh23:47:102
259950,0migration/232020-21kthreadcore22:05:202
259950,0migration/231706-21sh00:06:002
259950,0migration/230970-21latency_hist20:20:022
259950,0migration/229983-21gltestperf19:30:162
259950,0migration/227994-21kthreadcore19:50:212
259950,0migration/223297-21sh00:20:482
259950,0migration/22111-21fschecks_count23:30:142
259950,0migration/220359-21fschecks_time21:40:152
259950,0migration/210387-21sh22:55:062
214059950,0cyclictest0-21swapper/221:30:342
214059950,0cyclictest0-21swapper/200:27:472
214049955,0cyclictest0-21swapper/121:30:201
214049954,0cyclictest2271-21idleruntime-cro19:10:031
214049950,0cyclictest19524-21diskmemload23:24:191
214049950,0cyclictest0-21swapper/122:58:461
214039950,0cyclictest0-21swapper/022:25:150
179950,0migration/15511-21aten_r1power_fr20:25:141
179950,0migration/117986-21fschecks_time00:00:161
179950,0migration/115693-21kthreadcore21:55:201
179950,0migration/112937-21latency_hist20:55:021
179950,0migration/111686-21latency_hist21:15:021
169950,0watchdog/11219-21ssh21:05:251
169950,0watchdog/110524-21hddtemp_smartct23:35:161
149950,0migration/09805-21fschecks_time19:15:170
149950,0migration/08979-21kthreadcore20:50:190
149950,0migration/06493-21kthreadcore23:30:190
149950,0migration/04546-21kthreadcore22:10:180
149950,0migration/04391-21if_eth1.321:50:170
149950,0migration/03789-21sh21:10:140
149950,0migration/03317-21pmu-power19:55:230
149950,0migration/032693-21kthreadcore21:05:210
149950,0migration/031953-21sh23:45:460
149950,0migration/030577-21kthreadcore19:30:190
149950,0migration/029291-21kthreadcore23:25:190
149950,0migration/028812-21fschecks_time19:05:170
149950,0migration/02756-21cpuspeed_turbos19:10:150
149950,0migration/02733-21sh21:49:030
149950,0migration/026663-21cpuspeed_turbos23:05:140
149950,0migration/026635-21taskset22:03:380
149950,0migration/026040-21runrttasks23:04:510
149950,0migration/025278-21taskset22:20:360
149950,0migration/022458-21missed_timers00:20:210
149950,0migration/020319-21fschecks_time21:40:150
149950,0migration/019625-21kthreadcore19:45:220
149950,0migration/018607-21taskset21:58:340
149950,0migration/017027-21sh22:35:480
149950,0migration/016087-21unixbench_multi23:15:250
149950,0migration/015042-21latency23:35:200
149950,0migration/01423-21sendmail_mailtr21:25:240
149950,0migration/013403-21kthreadcore20:05:200
149950,0migration/013361-21kthreadcore00:15:190
149950,0migration/012535-21kthreadcore22:15:190
149950,0migration/01225-21kthreadcore20:20:190
339940,0migration/38736-21kthreadcore20:50:193
339940,0migration/330434-21kthreadcore19:05:203
29725040,0irq/27-eth0-tx-6559-21runrttasks19:25:140
29725040,0irq/27-eth0-tx-6559-21runrttasks19:25:140
29725040,0irq/27-eth0-tx-18405-21munin-node23:00:222
29725040,0irq/27-eth0-tx-17713-21aten_r1power_cu23:40:112
29725040,0irq/27-eth0-tx-11059-21munin-node22:15:162
29725040,0irq/27-eth0-tx-0-21swapper/223:55:432
29725040,0irq/27-eth0-tx-0-21swapper/223:51:282
29725040,0irq/27-eth0-tx-0-21swapper/223:25:192
29725040,0irq/27-eth0-tx-0-21swapper/223:10:152
29725040,0irq/27-eth0-tx-0-21swapper/223:05:032
29725040,0irq/27-eth0-tx-0-21swapper/222:35:582
29725040,0irq/27-eth0-tx-0-21swapper/222:00:412
29725040,0irq/27-eth0-tx-0-21swapper/221:55:132
29725040,0irq/27-eth0-tx-0-21swapper/221:46:202
29725040,0irq/27-eth0-tx-0-21swapper/221:25:212
29725040,0irq/27-eth0-tx-0-21swapper/221:20:072
29725040,0irq/27-eth0-tx-0-21swapper/221:15:172
29725040,0irq/27-eth0-tx-0-21swapper/221:10:052
29725040,0irq/27-eth0-tx-0-21swapper/200:12:392
29725040,0irq/27-eth0-tx-0-21swapper/020:45:250
29725040,0irq/27-eth0-tx-0-21swapper/019:50:200
29715040,0irq/26-eth0-rx-0-21swapper/323:35:143
29715040,0irq/26-eth0-rx-0-21swapper/323:15:543
29715040,0irq/26-eth0-rx-0-21swapper/322:45:103
29715040,0irq/26-eth0-rx-0-21swapper/322:36:323
29715040,0irq/26-eth0-rx-0-21swapper/321:10:303
29715040,0irq/26-eth0-rx-0-21swapper/320:00:223
29715040,0irq/26-eth0-rx-0-21swapper/300:10:133
259940,0migration/27073-21kthreadcore22:10:182
259940,0migration/218820-21gltestperf23:20:152
259940,0migration/214945-21kthreadcore23:35:202
259940,0migration/214610-21kthreadcore20:05:212
249940,0watchdog/29443-21cat19:35:252
249940,0watchdog/25802-21kthreadcore20:00:212
249940,0watchdog/216469-21kthreadcore20:30:212
249940,0watchdog/20-21swapper/222:50:202
249940,0watchdog/20-21swapper/200:00:162
214059944,0cyclictest0-21swapper/220:55:252
214059944,0cyclictest0-21swapper/220:25:182
214059943,0cyclictest3556-21lldpd19:28:232
214059943,0cyclictest3556-21lldpd19:28:232
214059943,0cyclictest3514-21snmpd22:25:542
214059943,0cyclictest3514-21snmpd21:50:092
214059943,0cyclictest3514-21snmpd20:14:392
214059943,0cyclictest3514-21snmpd19:42:242
214059943,0cyclictest3514-21snmpd19:23:232
214059943,0cyclictest3514-21snmpd19:08:232
214059943,0cyclictest31099-21awk19:55:172
214059943,0cyclictest22466-21awk19:00:172
214059943,0cyclictest19774-21awk20:35:182
214059943,0cyclictest11004-21awk23:15:172
214059943,0cyclictest10215-21awk19:15:192
214059940,0cyclictest27980-21grep20:40:192
214049943,0cyclictest3654-21awk22:30:171
214049943,0cyclictest3556-21lldpd23:25:261
214049943,0cyclictest3556-21lldpd22:44:551
214049943,0cyclictest3556-21lldpd19:56:231
214049943,0cyclictest3556-21lldpd19:41:241
214049943,0cyclictest3514-21snmpd22:11:541
214049943,0cyclictest3514-21snmpd21:48:091
214049943,0cyclictest3514-21snmpd00:10:251
214049943,0cyclictest30915-21sadc20:20:021
214049943,0cyclictest2982-21awk23:10:191
214049943,0cyclictest26661-21awk23:45:181
214049943,0cyclictest18749-21awk20:10:171
214049943,0cyclictest18622-21awk23:40:151
214049943,0cyclictest13953-21awk20:55:171
214049940,0cyclictest0-21swapper/122:24:431
214039944,0cyclictest0-21swapper/023:12:240
214039943,0cyclictest3514-21snmpd23:40:390
214039943,0cyclictest32682-21kworker/0:122:05:220
214039943,0cyclictest32682-21kworker/0:119:20:180
214039943,0cyclictest20691-21awk21:00:180
214039943,0cyclictest1903-21awk00:10:180
214039943,0cyclictest11105-21awk19:40:180
179940,0migration/14327-21kthreadcore20:45:211
179940,0migration/13799-21kthreadcore22:50:191
179940,0migration/131171-21aten_r1power_cu20:20:131
179940,0migration/128778-21kthreadcore00:25:201
179940,0migration/127810-21forks22:05:151
179940,0migration/127242-21sendmail_mailqu19:00:231
179940,0migration/126463-21irqcore00:05:181
179940,0migration/126003-21fschecks_time20:40:151
179940,0migration/123921-21taskset19:50:161
179940,0migration/121414-21kthreadcore21:40:191
179940,0migration/120037-21fschecks_time22:00:161
179940,0migration/116780-21diskmemload22:35:261
179940,0migration/116431-21taskset00:19:071
179940,0migration/116164-21cpuspeed_turbos19:20:141
179940,0migration/115981-21timerwakeupswit23:15:251
179940,0migration/113624-21kthreadcore19:15:211
169940,0watchdog/1618-21ssh23:07:001
169940,0watchdog/13371-21munin-node21:50:201
169940,0watchdog/127534-21hdparm22:45:161
169940,0watchdog/124749-21ssh23:01:041
169940,0watchdog/119974-21ssh21:20:131
169940,0watchdog/117155-21mdstat21:35:211
169940,0watchdog/10-21swapper/123:50:201
169940,0watchdog/10-21swapper/123:31:561
169940,0watchdog/10-21swapper/120:30:211
169940,0watchdog/10-21swapper/120:00:211
169940,0watchdog/10-21swapper/119:35:251
169940,0watchdog/10-21swapper/119:25:171
169940,0watchdog/10-21swapper/119:25:171
169940,0watchdog/10-21swapper/100:20:201
149940,0migration/09145-21runrttasks22:31:100
149940,0migration/04297-21kthreadcore23:50:200
149940,0migration/024557-21runrttasks00:03:330
149940,0migration/022892-21kthreadcore21:20:190
149940,0migration/014822-21missed_timers23:55:200
29725030,0irq/27-eth0-tx-12168-21aten2.4-expect20:30:140
29715030,0irq/26-eth0-rx-0-21swapper/320:47:123
29715030,0irq/26-eth0-rx-0-21swapper/320:30:033
29715030,0irq/26-eth0-rx-0-21swapper/320:19:123
29715030,0irq/26-eth0-rx-0-21swapper/320:10:223
29715030,0irq/26-eth0-rx-0-21swapper/320:05:153
29715030,0irq/26-eth0-rx-0-21swapper/319:52:013
29715030,0irq/26-eth0-rx-0-21swapper/319:49:413
29715030,0irq/26-eth0-rx-0-21swapper/319:33:543
29715030,0irq/26-eth0-rx-0-21swapper/319:15:223
29715030,0irq/26-eth0-rx-0-21swapper/319:00:153
214059932,0cyclictest3514-21snmpd20:20:092
214059932,0cyclictest17853-21awk19:45:202
214059930,0cyclictest2455-21latency_hist19:10:032
214049932,0cyclictest3556-21lldpd20:37:541
214049932,0cyclictest28901-21wc21:25:171
214049932,0cyclictest28429-21kthreadcore22:25:181
214049932,0cyclictest2488-21sensors19:30:241
214049932,0cyclictest22798-21cut21:00:201
214049932,0cyclictest10693-21irqstats23:55:181
214039933,0cyclictest32682-21kworker/0:122:50:030
214039933,0cyclictest32682-21kworker/0:122:45:090
214039933,0cyclictest32682-21kworker/0:122:40:110
214039933,0cyclictest32682-21kworker/0:121:30:080
214039933,0cyclictest32682-21kworker/0:121:15:080
214039933,0cyclictest32682-21kworker/0:120:55:160
214039933,0cyclictest32682-21kworker/0:120:10:140
214039933,0cyclictest32682-21kworker/0:120:00:040
214039933,0cyclictest32682-21kworker/0:119:35:040
214039933,0cyclictest19197-21kworker/0:223:20:030
214039933,0cyclictest19197-21kworker/0:200:25:090
214039933,0cyclictest19197-21kworker/0:200:05:030
179930,0migration/116862-21runrttasks19:45:081
179930,0migration/114145-21kthreadcore20:05:201
179930,0migration/110304-21diskmemload21:11:431
169930,0watchdog/10-21swapper/119:10:251
29715020,0irq/26-eth0-rx-0-21swapper/320:55:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional