You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-01 - 01:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Sep 01, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1582328584,0sleep00-21swapper/018:42:400
304450460,0irq/25-eth10-21swapper/318:41:123
1594123323,0sleep10-21swapper/118:44:011
1567322622,0sleep20-21swapper/218:40:542
161149976,0cyclictest11749-21fschecks_count22:45:251
8215060,0irq/24-0000:00:4427-21munin-run22:00:132
339960,0migration/3692-21sh00:01:013
339960,0migration/34557-21sh21:38:403
339960,0migration/328758-21sh20:51:013
339960,0migration/32328-21fschecks_count23:20:253
339960,0migration/321053-21sh20:47:063
339960,0migration/319244-21munin-node20:00:243
29715060,0irq/26-eth0-rx-19295-21python20:20:353
149960,0migration/0781-21kthreadcore19:45:310
149960,0migration/05088-21idleruntime-cro21:40:130
149960,0migration/03844-21sh21:58:280
149960,0migration/032449-21sh23:37:300
149960,0migration/027977-21kthreadcore20:05:280
149960,0migration/027465-21sh22:13:020
149960,0migration/019680-21latency_hist22:30:130
149960,0migration/010323-21sh00:10:220
8215050,0irq/24-0000:00:24699-21latency_hist19:40:132
8215050,0irq/24-0000:00:10067-21awk00:10:122
339950,0migration/36664-21df_abs21:00:253
339950,0migration/33838-21latency_hist19:00:133
339950,0migration/33457-21latency19:20:313
339950,0migration/33084-21kthreadcore21:15:303
339950,0migration/325025-21kthreadcore21:10:283
339950,0migration/32161-21irqcore20:35:303
339950,0migration/320912-21kthreadcore18:45:313
339950,0migration/320878-21irqcore22:30:283
339950,0migration/32034-21sh20:55:303
339950,0migration/317226-21munin-node19:30:353
339950,0migration/315457-21munin-node23:25:323
339950,0migration/315286-21munin-node21:05:283
339950,0migration/315028-21kthreadcore22:45:293
339950,0migration/314631-21kthreadcore21:45:303
339950,0migration/314139-21kthreadcore22:25:293
339950,0migration/314131-21taskset20:43:153
339950,0migration/311914-21kthreadcore23:05:293
339950,0migration/311163-21munin-node21:40:363
339950,0migration/310122-21diskmemload23:48:083
29725050,0irq/27-eth0-tx-4690-21munin-node22:00:282
29725050,0irq/27-eth0-tx-3348-21hwlatdetect23:00:262
29725050,0irq/27-eth0-tx-31433-21irqcore21:15:282
29725050,0irq/27-eth0-tx-31049-21kthreadcore21:35:282
29725050,0irq/27-eth0-tx-30496-21munin-node20:55:272
29725050,0irq/27-eth0-tx-22484-21munin-node21:10:312
29725050,0irq/27-eth0-tx-19155-21munin-node22:50:262
29725050,0irq/27-eth0-tx-13882-21tr23:25:292
29725050,0irq/27-eth0-tx-11915-21ssh22:25:212
29725050,0irq/27-eth0-tx-11900-21ssh21:41:482
29725050,0irq/27-eth0-tx-0-21swapper/223:57:032
29725050,0irq/27-eth0-tx-0-21swapper/223:49:282
29725050,0irq/27-eth0-tx-0-21swapper/223:42:072
29725050,0irq/27-eth0-tx-0-21swapper/222:58:522
29725050,0irq/27-eth0-tx-0-21swapper/221:50:312
29725050,0irq/27-eth0-tx-0-21swapper/019:29:510
29715050,0irq/26-eth0-rx-6406-21diskstats21:20:253
29715050,0irq/26-eth0-rx-4222-21if_err_eth1.322:40:283
29715050,0irq/26-eth0-rx-30925-21kthreadcore23:15:303
29715050,0irq/26-eth0-rx-24495-21sh23:32:283
29715050,0irq/26-eth0-rx-21375-21if_err_eth022:10:283
29715050,0irq/26-eth0-rx-19500-21if_eth1.323:10:283
29715050,0irq/26-eth0-rx-16965-21cut22:05:313
29715050,0irq/26-eth0-rx-16279-21cut21:25:303
29715050,0irq/26-eth0-rx-16259-21timerwakeupswit00:10:373
29715050,0irq/26-eth0-rx-0-21swapper/323:50:233
29715050,0irq/26-eth0-rx-0-21swapper/323:40:343
29715050,0irq/26-eth0-rx-0-21swapper/322:16:313
29715050,0irq/26-eth0-rx-0-21swapper/321:58:353
29715050,0irq/26-eth0-rx-0-21swapper/321:30:313
29715050,0irq/26-eth0-rx-0-21swapper/300:05:363
259950,0migration/29181-21timerandwakeup22:40:352
259950,0migration/230925-21munin-node18:55:292
259950,0migration/229835-21kthreadcore22:15:282
259950,0migration/229261-21sh20:52:282
259950,0migration/222365-21fschecks_count21:30:272
259950,0migration/220039-21sh22:09:332
259950,0migration/217773-21sh23:30:222
259950,0migration/217122-21kthreadcore20:45:282
259950,0migration/216124-21kthreadcore21:05:302
179950,0migration/127210-21unixbench-2d21:50:361
179950,0migration/124268-21irqrtprio18:50:301
179950,0migration/123733-21munin-node19:35:341
179950,0migration/11898-21sh22:36:501
179950,0migration/112525-21latency_hist22:05:131
161159954,0cyclictest3514-21snmpd00:11:472
161159954,0cyclictest25285-21grep22:30:312
161149955,0cyclictest14320-21diskmemload00:07:521
161149955,0cyclictest0-21swapper/123:43:261
161149955,0cyclictest0-21swapper/123:13:191
161149955,0cyclictest0-21swapper/121:40:471
161149950,0cyclictest0-21swapper/123:49:541
161149950,0cyclictest0-21swapper/123:33:021
161149950,0cyclictest0-21swapper/122:54:431
161149950,0cyclictest0-21swapper/122:15:181
161149950,0cyclictest0-21swapper/100:10:241
161129950,0cyclictest0-21swapper/021:50:440
169950,0watchdog/18733-21if_err_eth020:40:271
169950,0watchdog/15283-21kthreadcore19:00:321
169950,0watchdog/129777-21fschecks_time21:55:271
169950,0watchdog/125588-21kthreadcore20:50:311
149950,0migration/08177-21taskset23:20:530
149950,0migration/07674-21munin-node20:15:270
149950,0migration/07403-21kthreadcore19:00:320
149950,0migration/07339-21taskset20:36:250
149950,0migration/0473-21taskset20:30:520
149950,0migration/04204-21taskset21:15:370
149950,0migration/04120-21kthreadcore00:05:300
149950,0migration/03971-21munin-node19:20:360
149950,0migration/028775-21irqcore22:35:280
149950,0migration/024828-21kthreadcore21:10:280
149950,0migration/023475-21munin-node23:30:340
149950,0migration/020869-21fschecks_time20:25:260
149950,0migration/014668-21kthreadcore23:25:290
149950,0migration/013828-21irqrtprio22:05:290
149950,0migration/013778-21latency_hist21:25:130
149950,0migration/012626-21kthreadcore23:50:270
149950,0migration/01262-21taskset23:19:180
149950,0migration/011846-21cpuspeed_turbos19:30:260
149950,0migration/011831-21fschecks_time22:45:250
149950,0migration/010697-21kthreadcore19:50:310
339940,0migration/37943-21kthreadcore19:25:303
339940,0migration/35434-21kthreadcore23:00:293
339940,0migration/35315-21taskset19:46:233
339940,0migration/328983-21kthreadcore20:05:293
339940,0migration/328881-21kthreadcore19:40:313
339940,0migration/324791-21kthreadcore18:50:303
339940,0migration/324546-21runrttasks19:38:333
339940,0migration/320675-21df_inode20:25:263
339940,0migration/311799-21diskmemload22:03:363
339940,0migration/310621-21latency_hist19:05:133
29725040,0irq/27-eth0-tx-14320-21diskmemload21:25:132
29725040,0irq/27-eth0-tx-14139-21ssh21:45:282
29725040,0irq/27-eth0-tx-10654-21ssh23:05:242
29725040,0irq/27-eth0-tx-0-21swapper/223:50:502
29725040,0irq/27-eth0-tx-0-21swapper/223:36:132
29725040,0irq/27-eth0-tx-0-21swapper/222:45:192
29725040,0irq/27-eth0-tx-0-21swapper/222:35:262
29725040,0irq/27-eth0-tx-0-21swapper/020:45:330
29715040,0irq/26-eth0-rx-0-21swapper/323:55:233
29715040,0irq/26-eth0-rx-0-21swapper/323:37:363
29715040,0irq/26-eth0-rx-0-21swapper/322:56:283
29715040,0irq/26-eth0-rx-0-21swapper/322:50:293
29715040,0irq/26-eth0-rx-0-21swapper/322:35:253
29715040,0irq/26-eth0-rx-0-21swapper/322:21:443
29715040,0irq/26-eth0-rx-0-21swapper/321:50:323
29715040,0irq/26-eth0-rx-0-21swapper/319:51:483
29715040,0irq/26-eth0-rx-0-21swapper/319:00:303
259940,0migration/2894-21kthreadcore19:20:312
259940,0migration/27974-21kthreadcore19:00:322
259940,0migration/26364-21kthreadcore22:20:302
259940,0migration/25704-21kthreadcore23:20:292
259940,0migration/226344-21runrttasks20:26:142
259940,0migration/223388-21df_abs18:50:262
259940,0migration/220730-21kthreadcore23:10:292
259940,0migration/214423-21kthreadcore19:30:312
249940,0watchdog/231949-21users20:05:352
249940,0watchdog/225538-21kthreadcore22:10:312
249940,0watchdog/224557-21munin-node20:00:312
249940,0watchdog/212303-21ssh21:21:112
249940,0watchdog/20-21swapper/220:40:272
179940,0migration/18273-21kthreadcore21:20:291
179940,0migration/17912-21kthreadcore21:00:301
179940,0migration/17840-21kthreadcore22:40:301
179940,0migration/14403-21kthreadcore23:20:291
179940,0migration/131457-21kthreadcore18:55:291
179940,0migration/128623-21kthreadcore20:30:301
179940,0migration/127745-21taskset22:55:251
179940,0migration/127527-21kthreadcore19:15:301
179940,0migration/127242-21munin-node23:15:271
179940,0migration/126488-21taskset20:27:531
179940,0migration/126448-21fschecks_count20:05:251
179940,0migration/119734-21fschecks_count20:00:261
179940,0migration/117343-21runrttasks19:10:211
179940,0migration/113857-21kernelversion19:55:301
179940,0migration/112205-21kthreadcore23:05:291
179940,0migration/111614-21kthreadcore23:25:281
179940,0migration/111533-21latency_hist19:30:131
161159943,0cyclictest3556-21lldpd19:44:322
161159943,0cyclictest3514-21snmpd21:01:472
161159943,0cyclictest3514-21snmpd20:35:172
161159943,0cyclictest3514-21snmpd20:31:472
161159943,0cyclictest3514-21snmpd19:07:472
161159943,0cyclictest27816-21awk00:00:282
161159943,0cyclictest27391-21awk23:15:282
161159943,0cyclictest18306-21awk19:10:292
161159943,0cyclictest17210-21awk18:45:282
161159940,0cyclictest7739-21grep19:25:302
161149944,0cyclictest0-21swapper/123:56:221
161149943,0cyclictest5668-21awk22:00:271
161149943,0cyclictest3556-21lldpd22:25:331
161149943,0cyclictest3556-21lldpd22:21:331
161149943,0cyclictest3556-21lldpd20:23:321
161149943,0cyclictest3556-21lldpd19:09:021
161149943,0cyclictest3556-21lldpd18:45:321
161149943,0cyclictest32563-21awk19:45:301
161149943,0cyclictest30576-21awk21:35:261
161149943,0cyclictest26562-21awk23:35:261
161149943,0cyclictest25814-21awk19:40:281
161149943,0cyclictest20722-21awk22:30:271
161149943,0cyclictest15300-21awk21:05:281
161149943,0cyclictest14237-21awk21:45:291
161149943,0cyclictest12057-21awk23:50:261
161149943,0cyclictest1128-21awk20:10:281
161129944,0cyclictest0-21swapper/022:16:250
161129943,0cyclictest4960-21awk22:20:280
161129943,0cyclictest4084-21awk23:45:240
161129943,0cyclictest3556-21lldpd20:51:330
161129943,0cyclictest3514-21snmpd22:31:170
161129943,0cyclictest3514-21snmpd21:10:020
161129943,0cyclictest3241-21irqbalance23:11:510
161129943,0cyclictest3241-21irqbalance22:52:410
161129943,0cyclictest3241-21irqbalance21:43:110
161129943,0cyclictest30533-21awk18:55:280
161129943,0cyclictest29449-21sadc21:35:130
161129943,0cyclictest13410-21awk19:55:290
161129943,0cyclictest1049-21kworker/0:222:55:240
169940,0watchdog/15476-21sh20:58:511
169940,0watchdog/130455-21cut00:00:301
169940,0watchdog/122616-21munin-node21:10:231
169940,0watchdog/117796-21cut21:25:311
169940,0watchdog/112046-21kthreadcore20:15:311
169940,0watchdog/10-21swapper/123:02:311
169940,0watchdog/10-21swapper/122:10:311
169940,0watchdog/10-21swapper/121:15:271
149940,0migration/05197-21runrttasks20:58:080
149940,0migration/025492-21fschecks_time19:40:270
149940,0migration/019082-21kthreadcore19:10:310
149940,0migration/017789-21kthreadcore21:45:310
149940,0migration/013641-21sh21:03:460
149940,0migration/013010-21kthreadcore23:05:300
149940,0migration/012770-21sh21:22:360
149940,0migration/01146-21runrttasks00:02:160
29725030,0irq/27-eth0-tx-0-21swapper/019:15:350
29725030,0irq/27-eth0-tx-0-21swapper/019:05:240
29715030,0irq/26-eth0-rx-0-21swapper/320:30:153
29715030,0irq/26-eth0-rx-0-21swapper/320:15:253
29715030,0irq/26-eth0-rx-0-21swapper/320:10:253
29715030,0irq/26-eth0-rx-0-21swapper/319:55:263
29715030,0irq/26-eth0-rx-0-21swapper/319:15:243
259930,0migration/217925-21kthreadcore20:20:312
249930,0watchdog/20-21swapper/220:15:232
249930,0watchdog/20-21swapper/219:55:312
179930,0migration/110612-21kthreadcore19:50:311
161159932,0cyclictest5552-21grep20:10:312
161159932,0cyclictest24816-21awk19:15:282
161159931,0cyclictest31763-21cpu19:45:262
161149932,0cyclictest3556-21lldpd20:50:031
161149932,0cyclictest31497-21awk19:20:281
161149932,0cyclictest22693-21awk21:30:291
161129933,0cyclictest2642-21kworker/0:123:55:280
161129933,0cyclictest1049-21kworker/0:223:00:220
161129933,0cyclictest1049-21kworker/0:222:40:160
161129933,0cyclictest1049-21kworker/0:220:40:480
161129933,0cyclictest1049-21kworker/0:220:20:180
161129933,0cyclictest1049-21kworker/0:220:10:240
161129933,0cyclictest1049-21kworker/0:218:45:170
161129932,0cyclictest5644-21awk22:00:270
161129932,0cyclictest3241-21irqbalance23:42:510
161129932,0cyclictest23981-21awk18:50:290
161129932,0cyclictest19217-21awk19:35:290
161129930,0cyclictest19867-21perl20:00:260
169930,0watchdog/10-21swapper/120:35:271
339920,0migration/318354-21iostat_ios19:10:303
161159920,0cyclictest5846-21latency_hist19:50:132
161149921,0cyclictest4844-21latency_hist19:25:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional