You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 03:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Thu Jul 30, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2279826867,0sleep10-21swapper/118:44:321
304450600,0irq/25-eth10-21swapper/318:47:533
1678125049,0sleep00-21swapper/018:43:270
2295423425,0sleep20-21swapper/218:46:212
232029976,0cyclictest10905-21dump-pmu-power22:23:212
339960,0migration/34315-21runrttasks21:17:023
339960,0migration/331811-21sh23:14:013
339960,0migration/327748-21taskset20:48:523
339960,0migration/319063-21munin-node21:03:433
339960,0migration/317358-21sh22:46:013
29725060,0irq/27-eth0-tx-30089-21munin-node18:53:450
29715060,0irq/26-eth0-rx-30943-21cut20:53:373
29715060,0irq/26-eth0-rx-10761-21grep23:43:423
29715060,0irq/26-eth0-rx-10247-21munin-run22:43:213
29715060,0irq/26-eth0-rx-0-21swapper/322:54:553
259960,0migration/23536-21sh21:58:212
259960,0migration/227120-21sh21:29:012
232029965,0cyclictest2860-21crond22:18:212
232009966,0cyclictest0-21swapper/119:28:341
232009965,0cyclictest15972-21cut20:43:381
232009960,0cyclictest21383-21diskmemload21:02:421
232009960,0cyclictest0-21swapper/123:27:441
169960,0watchdog/17033-21nfs_client20:33:401
149960,0migration/028288-21sh21:32:370
149960,0migration/02623-21sh21:34:050
149960,0migration/014571-21kthreadcore21:23:390
339950,0migration/38902-21kthreadcore21:18:393
339950,0migration/37467-21kthreadcore20:58:383
339950,0migration/3550-21munin-node22:13:413
339950,0migration/35178-21idleruntime18:58:363
339950,0migration/34796-21kthreadcore22:18:393
339950,0migration/33673-21kthreadcore23:38:413
339950,0migration/33509-21kthreadcore20:08:393
339950,0migration/33441-21kthreadcore20:33:373
339950,0migration/331998-21munin-node19:18:373
339950,0migration/324884-21sh22:49:503
339950,0migration/323305-21kthreadcore21:08:383
339950,0migration/322912-21kthreadcore21:28:353
339950,0migration/321466-21kthreadcore22:08:373
339950,0migration/320280-21kthreadcore23:08:373
339950,0migration/315250-21kthreadcore21:43:373
339950,0migration/31481-21munin-node23:18:313
339950,0migration/314477-21kthreadcore19:28:403
339950,0migration/312053-21kthreadcore00:03:373
339950,0migration/312053-21kthreadcore00:03:373
29725050,0irq/27-eth0-tx-29415-21munin-node20:53:412
29725050,0irq/27-eth0-tx-26302-21munin-node22:53:382
29725050,0irq/27-eth0-tx-23411-21munin-node18:48:420
29725050,0irq/27-eth0-tx-18609-21cat21:03:392
29725050,0irq/27-eth0-tx-14845-21ssh00:06:182
29725050,0irq/27-eth0-tx-14845-21ssh00:06:182
29725050,0irq/27-eth0-tx-13038-21latency_hist21:03:222
29725050,0irq/27-eth0-tx-0-21swapper/222:36:232
29725050,0irq/27-eth0-tx-0-21swapper/221:36:172
29725050,0irq/27-eth0-tx-0-21swapper/221:23:002
29725050,0irq/27-eth0-tx-0-21swapper/020:01:340
29725050,0irq/27-eth0-tx-0-21swapper/019:08:410
29715050,0irq/26-eth0-rx-31864-21ssh23:36:173
29715050,0irq/26-eth0-rx-30110-21kthreadcore22:33:433
29715050,0irq/26-eth0-rx-3001-21irqrtprio22:58:373
29715050,0irq/26-eth0-rx-28820-21kthreadcore00:13:403
29715050,0irq/26-eth0-rx-25291-21if_eth1.223:53:373
29715050,0irq/26-eth0-rx-19955-21cut23:48:393
29715050,0irq/26-eth0-rx-18454-21ssh22:05:213
29715050,0irq/26-eth0-rx-0-21swapper/323:04:553
29715050,0irq/26-eth0-rx-0-21swapper/322:24:283
29715050,0irq/26-eth0-rx-0-21swapper/322:02:503
29715050,0irq/26-eth0-rx-0-21swapper/321:40:373
29715050,0irq/26-eth0-rx-0-21swapper/321:33:363
29715050,0irq/26-eth0-rx-0-21swapper/319:48:393
29715050,0irq/26-eth0-rx-0-21swapper/300:09:003
259950,0migration/29740-21sh22:41:532
259950,0migration/27665-21kthreadcore21:38:382
259950,0migration/228684-21kthreadcore23:13:382
259950,0migration/224160-21kthreadcore21:08:382
259950,0migration/223550-21sh23:51:482
259950,0migration/220448-21kthreadcore23:08:372
259950,0migration/216774-21kthreadcore19:53:392
259950,0migration/210247-21sh21:59:262
249950,0watchdog/218382-21ssh22:48:272
232029955,0cyclictest0-21swapper/221:52:372
232029955,0cyclictest0-21swapper/200:08:222
232029954,0cyclictest6957-21ssh23:39:402
232029950,0cyclictest0-21swapper/223:56:042
232029950,0cyclictest0-21swapper/223:37:372
232009955,0cyclictest0-21swapper/123:29:471
232009955,0cyclictest0-21swapper/122:58:441
232009955,0cyclictest0-21swapper/121:58:491
232009955,0cyclictest0-21swapper/100:06:221
232009955,0cyclictest0-21swapper/100:06:221
232009950,0cyclictest0-21swapper/122:34:181
232009950,0cyclictest0-21swapper/122:25:001
232009950,0cyclictest0-21swapper/122:21:131
232009950,0cyclictest0-21swapper/122:17:261
232009950,0cyclictest0-21swapper/121:44:001
232009950,0cyclictest0-21swapper/100:02:161
231999955,0cyclictest8430-21kworker/0:219:13:430
179950,0migration/15050-21munin-node21:18:331
179950,0migration/130429-21munin-node23:53:451
179950,0migration/123412-21munin-node18:48:321
179950,0migration/118443-21kthreadcore00:08:391
179950,0migration/115317-21kthreadcore19:03:391
169950,0watchdog/14708-21ssh21:18:121
149950,0migration/07168-21kthreadcore19:23:370
149950,0migration/06634-21kthreadcore21:18:390
149950,0migration/05853-21taskset19:46:330
149950,0migration/05129-21gltestperf18:58:360
149950,0migration/03618-21irqcore22:38:390
149950,0migration/03349-21taskset21:57:530
149950,0migration/031666-21taskset19:42:360
149950,0migration/026525-21taskset21:49:000
149950,0migration/025997-21taskset22:09:140
149950,0migration/025643-21kthreadcore00:13:390
149950,0migration/024366-21timerwakeupswit22:48:440
149950,0migration/017037-21aten_r1power_cu23:28:320
149950,0migration/016923-21munin-node19:03:440
149950,0migration/016353-21kthreadcore20:43:390
149950,0migration/016228-21kthreadcore22:23:390
149950,0migration/013920-21fschecks_time21:03:340
149950,0migration/013074-21munin-node19:53:330
149950,0migration/010621-21sh21:38:540
149950,0migration/010502-21kthreadcore20:13:410
149950,0migration/010304-21fschecks_count23:03:330
339940,0migration/3933-21taskset20:29:313
339940,0migration/318347-21kthreadcore20:43:403
339940,0migration/315658-21kthreadcore20:18:413
339940,0migration/310971-21kthreadcore20:13:413
29725040,0irq/27-eth0-tx-8437-21ssh23:43:332
29725040,0irq/27-eth0-tx-18750-21telnet19:33:330
29725040,0irq/27-eth0-tx-13008-21munin-node21:23:382
29725040,0irq/27-eth0-tx-12788-21forks21:43:322
29725040,0irq/27-eth0-tx-0-21swapper/223:28:412
29725040,0irq/27-eth0-tx-0-21swapper/223:23:352
29725040,0irq/27-eth0-tx-0-21swapper/223:03:432
29725040,0irq/27-eth0-tx-0-21swapper/222:30:442
29725040,0irq/27-eth0-tx-0-21swapper/222:23:372
29725040,0irq/27-eth0-tx-0-21swapper/222:03:232
29725040,0irq/27-eth0-tx-0-21swapper/220:49:122
29725040,0irq/27-eth0-tx-0-21swapper/020:28:410
29725040,0irq/27-eth0-tx-0-21swapper/019:18:320
29715040,0irq/26-eth0-rx-26094-21smartctl21:48:433
29715040,0irq/26-eth0-rx-0-21swapper/323:58:223
29715040,0irq/26-eth0-rx-0-21swapper/323:29:363
29715040,0irq/26-eth0-rx-0-21swapper/323:23:323
29715040,0irq/26-eth0-rx-0-21swapper/322:28:413
29715040,0irq/26-eth0-rx-0-21swapper/321:53:463
29715040,0irq/26-eth0-rx-0-21swapper/321:23:223
29715040,0irq/26-eth0-rx-0-21swapper/320:38:343
29715040,0irq/26-eth0-rx-0-21swapper/320:23:423
29715040,0irq/26-eth0-rx-0-21swapper/319:43:423
259940,0migration/25800-21kthreadcore18:58:382
259940,0migration/25702-21runrttasks23:58:442
259940,0migration/226038-21nfs4_client20:23:412
259940,0migration/22593-21kthreadcore19:18:392
259940,0migration/225491-21kthreadcore00:13:392
259940,0migration/223739-21runrttasks19:08:522
259940,0migration/222919-21kthreadcore19:33:392
259940,0migration/219840-21aten_r1power_po22:08:322
259940,0migration/215244-21kthreadcore19:03:392
259940,0migration/214579-21kthreadcore19:28:402
259940,0migration/21326-21kthreadcore18:53:412
249940,0watchdog/28626-21cut20:13:402
249940,0watchdog/227129-21if_err_eth1.220:03:362
249940,0watchdog/214617-21kthreadcore22:43:392
249940,0watchdog/210796-21ntp_kernel_pll_19:23:402
249940,0watchdog/20-21swapper/218:48:442
232029943,0cyclictest6831-21fschecks_time19:48:352
232029943,0cyclictest3514-21snmpd19:40:512
232029943,0cyclictest30358-21awk21:13:352
232029943,0cyclictest2417-21awk20:33:362
232029943,0cyclictest20550-21awk19:58:372
232029943,0cyclictest2019-21awk23:18:332
232029943,0cyclictest15811-21awk20:43:372
232029940,0cyclictest1856-21telnet22:58:322
232029940,0cyclictest1286-21munin-run20:33:222
232009943,0cyclictest9312-21awk23:43:401
232009943,0cyclictest9222-21awk20:38:371
232009943,0cyclictest8240-21awk20:13:391
232009943,0cyclictest7243-21awk19:48:371
232009943,0cyclictest6291-21awk19:23:361
232009943,0cyclictest485-21awk19:43:371
232009943,0cyclictest3556-21lldpd23:08:481
232009943,0cyclictest3556-21lldpd21:11:481
232009943,0cyclictest3514-21snmpd23:19:061
232009943,0cyclictest3514-21snmpd22:38:361
232009943,0cyclictest3514-21snmpd22:12:511
232009943,0cyclictest3514-21snmpd21:53:511
232009943,0cyclictest3514-21snmpd21:48:361
232009943,0cyclictest3514-21snmpd21:37:061
232009943,0cyclictest3514-21snmpd20:53:361
232009943,0cyclictest3514-21snmpd20:20:061
232009943,0cyclictest3514-21snmpd20:09:361
232009943,0cyclictest27226-21awk22:53:341
232009943,0cyclictest11511-21awk22:43:371
231999943,0cyclictest8743-21awk00:03:340
231999943,0cyclictest8743-21awk00:03:340
231999943,0cyclictest8430-21kworker/0:221:13:350
231999943,0cyclictest4824-21awk21:58:370
231999943,0cyclictest3556-21lldpd21:12:180
231999943,0cyclictest29024-21kworker/0:023:27:520
231999943,0cyclictest29024-21kworker/0:023:15:100
231999943,0cyclictest29024-21kworker/0:022:36:290
231999943,0cyclictest25242-21awk23:53:360
231999943,0cyclictest22480-21awk20:48:370
231999943,0cyclictest21540-21awk20:23:370
231999943,0cyclictest20005-21sadc21:48:210
231999943,0cyclictest19951-21awk22:28:360
231999943,0cyclictest17238-21awk23:48:360
231999942,0cyclictest29024-21kworker/0:023:38:400
231999940,0cyclictest0-21swapper/000:11:250
179940,0migration/1865-21kthreadcore18:53:411
179940,0migration/131847-21sh23:36:151
179940,0migration/131410-21taskset19:39:371
179940,0migration/129032-21kthreadcore20:28:381
179940,0migration/127994-21kthreadcore23:13:381
179940,0migration/127758-21kthreadcore20:03:381
179940,0migration/126264-21munin-node21:28:361
179940,0migration/123715-21latency22:48:391
179940,0migration/122109-21kthreadcore19:33:391
179940,0migration/122052-21kthreadcore20:23:391
179940,0migration/120871-21kernelversion19:58:381
179940,0migration/11879-21kthreadcore23:38:401
179940,0migration/117329-21munin-node22:03:411
179940,0migration/117027-21fschecks_time23:48:351
179940,0migration/110020-21processes18:58:421
169940,0watchdog/14624-21unixbench_multi19:18:441
169940,0watchdog/126788-21cut00:13:391
169940,0watchdog/123614-21unixbench_multi19:08:441
169940,0watchdog/118185-21kthreadcore21:23:401
169940,0watchdog/113751-21munin-node19:53:361
169940,0watchdog/10-21swapper/121:04:121
169940,0watchdog/10-21swapper/120:48:401
149940,0migration/09309-21irqcore20:38:370
149940,0migration/09072-21kthreadcore19:48:390
149940,0migration/04423-21kthreadcore23:18:350
149940,0migration/021663-21kthreadcore23:08:370
149940,0migration/016349-21kthreadcore22:03:390
149940,0migration/015566-21sh23:46:440
339930,0migration/33916-21taskset18:55:593
339930,0migration/319087-21diskstats19:33:353
29725030,0irq/27-eth0-tx-0-21swapper/020:03:330
29715030,0irq/26-eth0-rx-24794-21fschecks_count19:13:353
29715030,0irq/26-eth0-rx-0-21swapper/319:58:333
29715030,0irq/26-eth0-rx-0-21swapper/319:53:343
29715030,0irq/26-eth0-rx-0-21swapper/319:38:323
29715030,0irq/26-eth0-rx-0-21swapper/319:08:333
29715030,0irq/26-eth0-rx-0-21swapper/319:03:333
29715030,0irq/26-eth0-rx-0-21swapper/318:48:423
232029933,0cyclictest3556-21lldpd19:16:172
232029932,0cyclictest9198-21awk20:38:362
232029932,0cyclictest3556-21lldpd20:18:482
232029932,0cyclictest32475-21fschecks_count19:43:352
232029932,0cyclictest1438-21awk20:08:372
232009933,0cyclictest10685-21awk23:03:351
232009932,0cyclictest24819-21sensors22:28:431
232009930,0cyclictest0-21swapper/121:38:381
231999933,0cyclictest8430-21kworker/0:220:08:330
231999933,0cyclictest29024-21kworker/0:023:58:380
231999933,0cyclictest29024-21kworker/0:023:33:340
231999933,0cyclictest29024-21kworker/0:022:43:230
231999933,0cyclictest29024-21kworker/0:022:18:230
231999933,0cyclictest29024-21kworker/0:022:13:390
231999932,0cyclictest5361-21tr20:58:340
231999932,0cyclictest3514-21snmpd22:58:360
231999932,0cyclictest32628-21cut20:53:380
231999932,0cyclictest1651-21basename20:33:320
231999930,0cyclictest32052-21python22:53:410
231999930,0cyclictest14575-21perl20:18:370
231999930,0cyclictest12019-21munin-node19:28:390
169930,0watchdog/10-21swapper/119:13:361
339920,0migration/36363-21iostat_ios19:23:363
29715020,0irq/26-eth0-rx-0-21swapper/320:03:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional