You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 01:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Jan 11, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450710,0irq/25-eth10-21swapper/319:06:173
2538024230,0sleep10-21swapper/119:03:471
2551123526,0sleep20-21swapper/219:05:202
2534823221,0sleep00-21swapper/019:03:260
4160,0ktimersoftd/00-21swapper/021:59:290
4160,0ktimersoftd/00-21swapper/021:35:410
339960,0migration/3917-21if_eth1.321:52:323
339960,0migration/332174-21sh21:32:103
339960,0migration/331685-21sh22:11:473
339960,0migration/329306-21sh23:28:103
339960,0migration/322770-21sh22:45:513
339960,0migration/321025-21sh23:43:193
339960,0migration/318242-21fschecks_time21:02:313
339960,0migration/317484-21irqrtprio21:42:333
339960,0migration/313281-21sh23:38:593
29715060,0irq/26-eth0-rx-15319-21if_eth000:02:333
27160,0ktimersoftd/20-21swapper/221:59:292
257529960,0cyclictest0-21swapper/122:31:051
259960,0migration/224039-21latency_hist21:07:182
19160,0ktimersoftd/10-21swapper/121:59:291
149960,0migration/06746-21ntp_offset19:37:380
149960,0migration/020092-21unixbench_multi00:22:410
8215050,0irq/24-0000:00:32225-21latency_hist21:32:182
339950,0migration/39634-21kthreadcore23:57:343
339950,0migration/3921-21idleruntime21:32:313
339950,0migration/38824-21munin-node19:42:323
339950,0migration/37487-21taskset19:39:353
339950,0migration/37448-21diskmemload22:16:583
339950,0migration/34608-21kernelversion20:27:343
339950,0migration/328495-21kthreadcore22:47:343
339950,0migration/324790-21kthreadcore20:17:353
339950,0migration/323909-21fschecks_time23:07:303
339950,0migration/32190-21kthreadcore23:52:343
339950,0migration/320950-21fschecks_time19:27:313
339950,0migration/319393-21latency00:22:353
339950,0migration/315804-21fschecks_count23:02:313
339950,0migration/313428-21runrttasks00:21:153
339950,0migration/312595-21munin-node19:17:373
29725050,0irq/27-eth0-tx-6998-21munin-node19:17:302
29725050,0irq/27-eth0-tx-6401-21munin-node23:57:412
29725050,0irq/27-eth0-tx-31503-21munin-node22:52:392
29725050,0irq/27-eth0-tx-30650-21ssh22:29:292
29725050,0irq/27-eth0-tx-23992-21iostat23:27:332
29725050,0irq/27-eth0-tx-17436-21cut00:02:352
29725050,0irq/27-eth0-tx-15196-21fschecks_count23:42:292
29725050,0irq/27-eth0-tx-14960-21munin-node23:22:282
29725050,0irq/27-eth0-tx-0-21swapper/223:38:092
29725050,0irq/27-eth0-tx-0-21swapper/222:51:472
29725050,0irq/27-eth0-tx-0-21swapper/221:56:002
29725050,0irq/27-eth0-tx-0-21swapper/221:50:432
29725050,0irq/27-eth0-tx-0-21swapper/221:09:572
29725050,0irq/27-eth0-tx-0-21swapper/200:19:542
29725050,0irq/27-eth0-tx-0-21swapper/200:11:532
29715050,0irq/26-eth0-rx-31444-21rm21:50:343
29715050,0irq/26-eth0-rx-30484-21ssh21:08:043
29715050,0irq/26-eth0-rx-29662-21ssh22:27:383
29715050,0irq/26-eth0-rx-2234-21kthreadcore00:12:343
29715050,0irq/26-eth0-rx-22267-21cpuspeed_turbos00:27:283
29715050,0irq/26-eth0-rx-2188-21kthreadcore21:12:333
29715050,0irq/26-eth0-rx-19215-21kthreadcore22:02:353
29715050,0irq/26-eth0-rx-13147-21cut21:37:343
29715050,0irq/26-eth0-rx-0-21swapper/323:52:133
29715050,0irq/26-eth0-rx-0-21swapper/323:24:123
29715050,0irq/26-eth0-rx-0-21swapper/323:15:003
29715050,0irq/26-eth0-rx-0-21swapper/322:57:143
29715050,0irq/26-eth0-rx-0-21swapper/322:37:233
29715050,0irq/26-eth0-rx-0-21swapper/321:57:313
29715050,0irq/26-eth0-rx-0-21swapper/321:24:593
29715050,0irq/26-eth0-rx-0-21swapper/321:17:443
29715050,0irq/26-eth0-rx-0-21swapper/300:32:353
257539955,0cyclictest0-21swapper/222:12:342
257539950,0cyclictest0-21swapper/222:22:402
257529955,0cyclictest0-21swapper/123:54:571
257529955,0cyclictest0-21swapper/121:35:071
257529955,0cyclictest0-21swapper/121:21:521
257529953,0cyclictest6913-21awk00:17:311
257529950,0cyclictest0-21swapper/122:51:251
257529950,0cyclictest0-21swapper/122:41:391
257529950,0cyclictest0-21swapper/122:04:071
257529950,0cyclictest0-21swapper/121:45:031
257529950,0cyclictest0-21swapper/100:25:421
257529950,0cyclictest0-21swapper/100:14:491
257519955,0cyclictest0-21swapper/022:24:500
257519955,0cyclictest0-21swapper/022:13:200
257519950,0cyclictest0-21swapper/022:18:320
259950,0migration/25373-21open_inodes21:12:372
259950,0migration/231215-21latency_hist23:12:172
259950,0migration/222060-21sh23:04:302
259950,0migration/220068-21kthreadcore21:42:342
259950,0migration/215888-21unixbench_multi20:32:412
259950,0migration/214631-21sh23:22:072
249950,0watchdog/213789-21sensors_volt21:17:402
179950,0migration/17633-21fschecks_time23:17:311
179950,0migration/14660-21needreboot22:52:351
179950,0migration/1346-21fschecks_time21:12:301
179950,0migration/132044-21latency_hist22:12:171
179950,0migration/129224-21kthreadcore19:32:341
179950,0migration/126217-21kthreadcore21:07:331
149950,0migration/09018-21kthreadcore22:57:330
149950,0migration/08099-21aten_r1power_fr21:17:290
149950,0migration/08096-21kthreadcore20:27:350
149950,0migration/07436-21taskset21:56:240
149950,0migration/07162-21taskset22:36:570
149950,0migration/06312-21kthreadcore20:52:330
149950,0migration/0489-21vmstat19:32:410
149950,0migration/0439-21kthreadcore00:32:350
149950,0migration/032601-21kthreadcore23:52:330
149950,0migration/031788-21kthreadcore00:12:330
149950,0migration/031704-21latency_hist22:32:170
149950,0migration/029999-21timerandwakeup22:07:410
149950,0migration/029047-21munin-node22:47:370
149950,0migration/026366-21taskset19:28:240
149950,0migration/025422-21irqcore21:07:330
149950,0migration/023998-21munin-node23:27:330
149950,0migration/02326-21switchtime20:22:410
149950,0migration/023124-21fschecks_time00:07:290
149950,0migration/022997-21cpuspeed_turbos23:47:290
149950,0migration/022641-21sh21:43:230
149950,0migration/021940-21munin-node21:22:400
149950,0migration/021752-21taskset20:13:230
149950,0migration/021272-21latency_hist19:52:180
149950,0migration/02041-21kthreadcore21:12:330
149950,0migration/017888-21kthreadcore23:02:350
149950,0migration/017095-21taskset20:58:500
149950,0migration/015859-21cpuspeed_turbos22:42:290
149950,0migration/015615-21fschecks_time23:22:290
149950,0migration/015029-21fschecks_time00:02:310
149950,0migration/010114-21aten_r1power_po20:32:280
8215040,0irq/24-0000:00:20053-21latency_hist19:27:182
8215040,0irq/24-0000:00:0-21swapper/219:42:182
339940,0migration/36508-21sh22:35:053
339940,0migration/316942-21irqrtprio20:12:353
339940,0migration/312446-21missed_timers23:17:363
339940,0migration/312292-21kthreadcore20:32:343
29725040,0irq/27-eth0-tx-8210-21munin-node21:37:322
29725040,0irq/27-eth0-tx-24036-21diskmemload22:42:302
29725040,0irq/27-eth0-tx-24036-21diskmemload21:22:232
29725040,0irq/27-eth0-tx-23310-21munin-node20:42:302
29725040,0irq/27-eth0-tx-1458-21tail23:52:332
29725040,0irq/27-eth0-tx-0-21swapper/223:47:202
29725040,0irq/27-eth0-tx-0-21swapper/223:07:232
29725040,0irq/27-eth0-tx-0-21swapper/222:32:202
29725040,0irq/27-eth0-tx-0-21swapper/222:07:482
29725040,0irq/27-eth0-tx-0-21swapper/219:47:332
29725040,0irq/27-eth0-tx-0-21swapper/200:27:432
29725040,0irq/27-eth0-tx-0-21swapper/200:12:282
29715040,0irq/26-eth0-rx-14026-21df_inode19:22:303
29715040,0irq/26-eth0-rx-0-21swapper/323:33:073
29715040,0irq/26-eth0-rx-0-21swapper/322:59:423
29715040,0irq/26-eth0-rx-0-21swapper/322:24:523
29715040,0irq/26-eth0-rx-0-21swapper/322:17:213
29715040,0irq/26-eth0-rx-0-21swapper/320:37:353
29715040,0irq/26-eth0-rx-0-21swapper/319:52:313
29715040,0irq/26-eth0-rx-0-21swapper/319:07:353
29715040,0irq/26-eth0-rx-0-21swapper/300:07:263
257539944,0cyclictest0-21swapper/200:32:252
257539943,0cyclictest962-21awk21:32:322
257539943,0cyclictest8443-21awk22:37:332
257539943,0cyclictest5231-21awk20:52:322
257539943,0cyclictest3556-21lldpd22:57:312
257539943,0cyclictest3556-21lldpd20:22:312
257539943,0cyclictest3556-21lldpd19:57:312
257539943,0cyclictest3556-21lldpd19:53:312
257539943,0cyclictest3556-21lldpd19:40:312
257539943,0cyclictest3514-21snmpd20:57:252
257539943,0cyclictest3514-21snmpd19:23:252
257539943,0cyclictest31938-21awk23:32:312
257539943,0cyclictest3134-21awk20:02:332
257539943,0cyclictest30851-21awk20:47:322
257539943,0cyclictest16662-21awk20:12:342
257539940,0cyclictest7828-21munin-node22:17:282
257529944,0cyclictest0-21swapper/123:41:431
257529944,0cyclictest0-21swapper/123:22:191
257529944,0cyclictest0-21swapper/100:32:191
257529943,0cyclictest888-21awk21:52:321
257529943,0cyclictest8840-21awk19:42:321
257529943,0cyclictest7282-21sadc22:37:171
257529943,0cyclictest3556-21lldpd20:16:011
257529943,0cyclictest3514-21snmpd23:47:401
257529943,0cyclictest3514-21snmpd22:57:401
257529943,0cyclictest3514-21snmpd19:52:401
257529943,0cyclictest3514-21snmpd19:23:401
257529943,0cyclictest3514-21snmpd19:18:401
257529943,0cyclictest32234-21awk23:12:321
257529943,0cyclictest26899-21awk19:07:321
257529943,0cyclictest25283-21awk21:27:321
257529943,0cyclictest24236-21awk23:07:321
257529943,0cyclictest2039-21awk19:37:321
257529943,0cyclictest17567-21awk20:37:331
257529943,0cyclictest15600-21awk23:42:311
257519943,0cyclictest7931-21awk23:17:320
257519943,0cyclictest3556-21lldpd23:45:020
257519943,0cyclictest3514-21snmpd23:10:550
257519943,0cyclictest3514-21snmpd21:48:100
257519943,0cyclictest25251-21awk21:27:320
257519943,0cyclictest14565-21awk19:22:330
257519943,0cyclictest14515-21kworker/0:019:17:310
257519940,0cyclictest6782-21basename19:17:180
259940,0migration/25779-21kthreadcore20:27:342
259940,0migration/24989-21kthreadcore19:12:332
259940,0migration/228315-21kthreadcore19:07:342
259940,0migration/218953-21kthreadcore22:02:342
259940,0migration/216653-21aten_r1power_cu20:37:282
249940,0watchdog/217897-21cut00:22:352
249940,0watchdog/20-21swapper/220:17:362
179940,0migration/17028-21kthreadcore20:02:351
179940,0migration/16850-21kthreadcore20:52:331
179940,0migration/132654-21kthreadcore20:47:341
179940,0migration/13257-21latency_hist20:27:181
179940,0migration/131463-21fschecks_count23:32:291
179940,0migration/129590-21taskset20:46:031
179940,0migration/127182-21kthreadcore21:47:341
179940,0migration/127159-21kthreadcore20:17:361
179940,0migration/125337-21kthreadcore00:27:331
179940,0migration/122125-21timerwakeupswit21:22:411
179940,0migration/119161-21kthreadcore22:22:351
179940,0migration/115705-21aten_r1power_vo22:42:281
179940,0migration/115678-21irqcore19:47:331
179940,0migration/114883-21fschecks_count00:02:311
179940,0migration/112328-21kthreadcore20:07:371
179940,0migration/110039-21kthreadcore21:37:331
169940,0watchdog/16559-21runrttasks23:02:191
169940,0watchdog/129861-21ssh23:29:471
169940,0watchdog/129279-21kthreadcore19:57:361
169940,0watchdog/122392-21ssh00:07:231
169940,0watchdog/112840-21cut20:57:361
169940,0watchdog/112173-21python23:57:391
169940,0watchdog/11106-21kthreadcore20:22:361
169940,0watchdog/10-21swapper/119:12:281
149940,0migration/0952-21kthreadcore23:12:340
149940,0migration/09308-21kthreadcore23:37:350
149940,0migration/09075-21kthreadcore23:57:330
149940,0migration/08850-21irqrtprio22:37:340
149940,0migration/05168-21sh23:33:440
149940,0migration/023326-21kernelversion00:27:330
149940,0migration/023027-21sh22:04:590
149940,0migration/022884-21latency21:02:360
149940,0migration/010840-21kthreadcore19:42:350
149940,0migration/010399-21kthreadcore21:37:330
8215030,0irq/24-0000:00:0-21swapper/220:07:182
339930,0migration/32835-21runrttasks20:25:083
29715030,0irq/26-eth0-rx-0-21swapper/320:57:303
29715030,0irq/26-eth0-rx-0-21swapper/320:52:403
29715030,0irq/26-eth0-rx-0-21swapper/320:49:073
29715030,0irq/26-eth0-rx-0-21swapper/320:42:283
29715030,0irq/26-eth0-rx-0-21swapper/320:07:303
29715030,0irq/26-eth0-rx-0-21swapper/320:03:573
29715030,0irq/26-eth0-rx-0-21swapper/319:47:283
29715030,0irq/26-eth0-rx-0-21swapper/319:33:333
29715030,0irq/26-eth0-rx-0-21swapper/319:12:273
257539932,0cyclictest27923-21awk19:32:322
257529933,0cyclictest3556-21lldpd21:03:011
257529932,0cyclictest8767-21awk22:17:301
257529932,0cyclictest21170-21awk19:27:331
257529932,0cyclictest10885-21awk20:32:321
257519933,0cyclictest478-21kworker/0:222:52:260
257519933,0cyclictest3556-21lldpd00:17:320
257519933,0cyclictest14515-21kworker/0:020:47:290
257519933,0cyclictest14515-21kworker/0:020:42:190
257519933,0cyclictest14515-21kworker/0:020:37:330
257519933,0cyclictest14515-21kworker/0:020:17:190
257519933,0cyclictest14515-21kworker/0:020:07:190
257519933,0cyclictest14515-21kworker/0:020:02:190
257519933,0cyclictest14515-21kworker/0:019:57:190
257519933,0cyclictest14515-21kworker/0:019:48:050
257519933,0cyclictest14515-21kworker/0:019:07:190
169930,0watchdog/10-21swapper/122:09:271
29715020,0irq/26-eth0-rx-0-21swapper/319:57:263
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional