You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 10:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Jan 07, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450650,0irq/25-eth10-21swapper/319:07:083
452023524,0sleep00-21swapper/019:06:590
445823524,0sleep20-21swapper/219:06:172
430823026,0sleep10-21swapper/119:04:321
46489960,0cyclictest2828-21diskmemload21:35:282
46479960,0cyclictest0-21swapper/121:25:541
35160,0ktimersoftd/30-21swapper/323:46:333
339960,0migration/3684-21munin-node19:52:533
339960,0migration/331157-21sh22:21:063
339960,0migration/324840-21taskset22:56:033
339960,0migration/32467-21sh23:40:343
339960,0migration/313092-21sh00:26:313
339960,0migration/311985-21sh23:47:463
29715060,0irq/26-eth0-rx-13990-21fschecks_time21:32:533
149960,0migration/09312-21sh23:06:250
149960,0migration/030533-21munin-node22:38:000
149960,0migration/028299-21sh00:15:170
149960,0migration/016030-21timerwakeupswit23:08:020
8215050,0irq/24-0000:00:818-21cat20:17:352
8215050,0irq/24-0000:00:25490-21latency_hist22:57:402
46489955,0cyclictest0-21swapper/223:44:162
46489955,0cyclictest0-21swapper/222:31:542
46489954,0cyclictest3514-21snmpd20:26:462
46489950,0cyclictest0-21swapper/223:05:292
46489950,0cyclictest0-21swapper/221:39:522
46489950,0cyclictest0-21swapper/221:30:342
46479955,0cyclictest2828-21diskmemload23:51:151
46479955,0cyclictest0-21swapper/123:42:451
46479955,0cyclictest0-21swapper/122:52:551
46479955,0cyclictest0-21swapper/122:47:151
46479954,0cyclictest29906-21ssh21:42:361
46479950,0cyclictest0-21swapper/123:55:211
46479950,0cyclictest0-21swapper/122:49:081
46469955,0cyclictest0-21swapper/021:10:250
46469955,0cyclictest0-21swapper/000:12:120
46469950,0cyclictest5645-21basename19:32:400
46469950,0cyclictest0-21swapper/021:27:270
46469950,0cyclictest0-21swapper/000:33:570
339950,0migration/39126-21taskset23:23:423
339950,0migration/38947-21aten_r1power_vo20:47:503
339950,0migration/37164-21runrttasks22:43:333
339950,0migration/35640-21taskset19:07:553
339950,0migration/34135-21munin-node21:07:563
339950,0migration/33845-21munin-node21:43:013
339950,0migration/329118-21fschecks_count23:57:513
339950,0migration/325225-21kthreadcore21:57:583
339950,0migration/32437-21kthreadcore19:27:573
339950,0migration/322323-21munin-node21:37:533
339950,0migration/319153-21if_eth1.219:17:573
339950,0migration/317176-21sendmail_mailst21:13:013
339950,0migration/316055-21sh22:49:533
339950,0migration/315360-21munin-node20:27:553
339950,0migration/313875-21munin-node20:02:543
339950,0migration/310532-21kthreadcore22:27:573
29725050,0irq/27-eth0-tx-8410-21ps22:07:562
29725050,0irq/27-eth0-tx-19201-21munin-node19:43:042
29725050,0irq/27-eth0-tx-1657-21kthreadcore22:22:572
29725050,0irq/27-eth0-tx-14698-21nfs_client22:47:592
29725050,0irq/27-eth0-tx-10118-21ssh23:26:072
29725050,0irq/27-eth0-tx-0-21swapper/223:16:082
29725050,0irq/27-eth0-tx-0-21swapper/222:46:462
29725050,0irq/27-eth0-tx-0-21swapper/221:58:332
29715050,0irq/26-eth0-rx-8199-21kthreadcore00:02:573
29715050,0irq/26-eth0-rx-7968-21grep21:27:573
29715050,0irq/26-eth0-rx-7168-21if_err_eth1.321:47:553
29715050,0irq/26-eth0-rx-30575-21interrupts00:17:553
29715050,0irq/26-eth0-rx-25886-21open_files23:53:003
29715050,0irq/26-eth0-rx-2399-21kthreadcore20:17:563
29715050,0irq/26-eth0-rx-1976-21kthreadcore22:22:573
29715050,0irq/26-eth0-rx-16084-21kthreadcore00:07:573
29715050,0irq/26-eth0-rx-15713-21grep00:27:563
29715050,0irq/26-eth0-rx-14576-21aten_r1power_cu21:52:513
29715050,0irq/26-eth0-rx-14486-21kthreadcore23:27:573
29715050,0irq/26-eth0-rx-0-21swapper/323:32:563
29715050,0irq/26-eth0-rx-0-21swapper/323:20:353
29715050,0irq/26-eth0-rx-0-21swapper/322:59:583
29715050,0irq/26-eth0-rx-0-21swapper/322:32:583
29715050,0irq/26-eth0-rx-0-21swapper/322:09:263
29715050,0irq/26-eth0-rx-0-21swapper/322:07:013
29715050,0irq/26-eth0-rx-0-21swapper/321:23:393
29715050,0irq/26-eth0-rx-0-21swapper/321:22:263
29715050,0irq/26-eth0-rx-0-21swapper/300:32:523
29715050,0irq/26-eth0-rx-0-21swapper/300:15:503
259950,0migration/232386-21sh22:59:052
259950,0migration/214747-21cpuspeed_turbos20:27:532
249950,0watchdog/220948-21perl23:52:542
179950,0migration/16529-21fschecks_time19:32:531
179950,0migration/130178-21kthreadcore23:37:571
179950,0migration/126527-21fschecks_time19:47:531
179950,0migration/126284-21sh23:17:211
179950,0migration/112843-21kthreadcore23:27:561
179950,0migration/111684-21kthreadcore22:27:571
179950,0migration/110572-21munin-node21:28:011
149950,0migration/09554-21kthreadcore21:27:570
149950,0migration/07463-21kthreadcore00:22:550
149950,0migration/0743-21kthreadcore22:02:560
149950,0migration/06422-21kthreadcore23:22:570
149950,0migration/05697-21kthreadcore22:42:570
149950,0migration/05664-21sh21:46:580
149950,0migration/04699-21munin-node23:42:550
149950,0migration/030089-21kthreadcore23:37:570
149950,0migration/029503-21sh21:41:360
149950,0migration/026695-21latency_hist20:12:400
149950,0migration/024984-21kthreadcore21:17:590
149950,0migration/024858-21kthreadcore21:57:580
149950,0migration/023870-21kthreadcore20:57:590
149950,0migration/019783-21kthreadcore19:17:580
149950,0migration/018555-21kthreadcore21:52:580
149950,0migration/018331-21kthreadcore22:32:540
149950,0migration/017401-21latency_hist22:52:400
149950,0migration/016538-21kthreadcore20:02:580
149950,0migration/01599-21sh23:20:560
149950,0migration/015944-21kthreadcore23:47:550
149950,0migration/014684-21taskset22:11:240
149950,0migration/013948-21kthreadcore19:12:570
149950,0migration/013192-21kthreadcore22:47:570
149950,0migration/012748-21kthreadcore22:27:570
149950,0migration/012513-21taskset21:48:210
149950,0migration/012512-21kthreadcore20:23:000
149950,0migration/012024-21taskset19:34:220
149950,0migration/01064-21sh23:01:480
8215040,0irq/24-0000:00:20974-21latency_hist00:12:392
46489943,0cyclictest3556-21lldpd20:48:212
46489943,0cyclictest3241-21irqbalance00:22:582
46489943,0cyclictest28643-21awk20:37:542
46489943,0cyclictest20882-21awk21:17:562
46489943,0cyclictest19157-21awk19:17:572
46489941,0cyclictest0-21swapper/219:32:512
46489940,0cyclictest22112-21munin-node20:57:582
46479944,0cyclictest0-21swapper/122:25:291
46479943,0cyclictest3556-21lldpd23:36:541
46479943,0cyclictest3514-21snmpd22:32:471
46479943,0cyclictest3514-21snmpd22:11:311
46479943,0cyclictest3514-21snmpd20:58:311
46479943,0cyclictest3514-21snmpd20:42:461
46479943,0cyclictest3514-21snmpd20:32:461
46479943,0cyclictest3514-21snmpd20:27:461
46479943,0cyclictest3514-21snmpd20:24:461
46479943,0cyclictest3514-21snmpd00:07:471
46479943,0cyclictest3514-21snmpd00:04:021
46479943,0cyclictest25674-21awk22:37:541
46479943,0cyclictest21134-21awk20:07:561
46479943,0cyclictest15426-21awk21:52:551
46479943,0cyclictest14857-21awk00:27:551
46479943,0cyclictest13531-21awk19:37:561
46479943,0cyclictest12321-21awk21:12:551
46469944,0cyclictest0-21swapper/023:59:350
46469943,0cyclictest3556-21lldpd20:29:510
46469943,0cyclictest3514-21snmpd23:28:170
46469943,0cyclictest3241-21irqbalance22:25:180
46469943,0cyclictest28658-21awk20:37:540
46469943,0cyclictest19576-21grep23:12:560
339940,0migration/37825-21taskset20:22:523
339940,0migration/36368-21kthreadcore20:42:563
339940,0migration/327639-21kthreadcore19:47:563
339940,0migration/320164-21kthreadcore20:52:583
29725040,0irq/27-eth0-tx-2743-21ssh00:18:002
29725040,0irq/27-eth0-tx-19357-21munin-node23:32:552
29725040,0irq/27-eth0-tx-0-21swapper/223:27:522
29725040,0irq/27-eth0-tx-0-21swapper/223:17:412
29725040,0irq/27-eth0-tx-0-21swapper/223:08:312
29725040,0irq/27-eth0-tx-0-21swapper/222:37:412
29725040,0irq/27-eth0-tx-0-21swapper/222:18:272
29725040,0irq/27-eth0-tx-0-21swapper/222:12:512
29725040,0irq/27-eth0-tx-0-21swapper/222:03:002
29725040,0irq/27-eth0-tx-0-21swapper/221:47:522
29725040,0irq/27-eth0-tx-0-21swapper/221:44:002
29725040,0irq/27-eth0-tx-0-21swapper/221:22:572
29725040,0irq/27-eth0-tx-0-21swapper/220:07:562
29725040,0irq/27-eth0-tx-0-21swapper/219:56:532
29725040,0irq/27-eth0-tx-0-21swapper/200:27:432
29725040,0irq/27-eth0-tx-0-21swapper/200:07:432
29725040,0irq/27-eth0-tx-0-21swapper/200:03:132
29715040,0irq/26-eth0-rx-0-21swapper/323:12:533
29715040,0irq/26-eth0-rx-0-21swapper/323:09:113
29715040,0irq/26-eth0-rx-0-21swapper/323:02:413
29715040,0irq/26-eth0-rx-0-21swapper/322:38:023
29715040,0irq/26-eth0-rx-0-21swapper/322:12:473
29715040,0irq/26-eth0-rx-0-21swapper/320:07:543
259940,0migration/25228-21munin-node19:28:052
259940,0migration/23179-21aten_r1power_fr21:07:522
259940,0migration/230286-21kthreadcore23:57:542
259940,0migration/226367-21fschecks_count19:47:532
259940,0migration/223039-21kthreadcore20:32:562
259940,0migration/220337-21sendmail_mailtr21:53:032
259940,0migration/218483-21kthreadcore22:32:542
259940,0migration/212760-21gltestperf23:47:512
259940,0migration/210142-21kthreadcore19:57:572
249940,0watchdog/27096-21unixbench_multi20:18:022
249940,0watchdog/231643-21kthreadcore23:37:582
249940,0watchdog/22131-21munin-node20:42:502
249940,0watchdog/217624-21kthreadcore20:02:582
249940,0watchdog/20-21swapper/221:12:462
249940,0watchdog/20-21swapper/219:37:592
249940,0watchdog/20-21swapper/219:07:552
249940,0watchdog/20-21swapper/200:32:532
179940,0migration/18516-21munin-node23:23:021
179940,0migration/18387-21sh23:04:081
179940,0migration/16281-21kthreadcore21:07:581
179940,0migration/1453-21taskset20:13:421
179940,0migration/132071-21kthreadcore21:42:551
179940,0migration/128776-21taskset21:02:491
179940,0migration/128753-21aten_r1power_vo23:57:501
179940,0migration/127060-21df_abs23:17:531
179940,0migration/127004-21kthreadcore22:17:571
179940,0migration/123078-21if_eth000:32:551
179940,0migration/12258-21kthreadcore22:02:571
179940,0migration/118242-21kthreadcore20:52:571
179940,0migration/116564-21irqcore22:12:551
179940,0migration/115379-21kthreadcore21:32:561
169940,0watchdog/122352-21kthreadcore21:17:581
169940,0watchdog/12201-21cut00:17:571
169940,0watchdog/121980-21kthreadcore19:17:591
169940,0watchdog/10-21swapper/123:07:421
169940,0watchdog/10-21swapper/122:57:581
169940,0watchdog/10-21swapper/119:22:591
169940,0watchdog/10-21swapper/100:22:531
149940,0migration/09665-21kthreadcore19:07:590
149940,0migration/03091-21timerandwakeup00:18:020
149940,0migration/026735-21kthreadcore19:23:010
149940,0migration/024553-21kthreadcore23:32:570
149940,0migration/020539-21cpuspeed_turbos23:52:520
149940,0migration/01783-21fschecks_time20:17:530
149940,0migration/016054-21kthreadcore21:12:580
149940,0migration/012543-21sensors_volt19:58:020
46489932,0cyclictest3556-21lldpd21:02:512
46489932,0cyclictest16408-21awk20:52:552
46489930,0cyclictest0-21swapper/219:12:582
46479932,0cyclictest3514-21snmpd21:47:461
46479932,0cyclictest31652-21basename19:27:521
46479930,0cyclictest0-21swapper/120:48:411
46469932,0cyclictest8565-21sadc20:47:400
46469932,0cyclictest3556-21lldpd22:17:530
46469932,0cyclictest3241-21irqbalance22:12:580
46469932,0cyclictest29687-21awk21:02:550
46469932,0cyclictest26809-21sensors20:33:010
46469932,0cyclictest26670-21awk19:47:540
46469932,0cyclictest20470-21irqstats19:42:560
46469932,0cyclictest16345-21awk20:52:550
46469932,0cyclictest14316-21awk21:32:540
46469932,0cyclictest12695-21basename19:37:520
46469932,0cyclictest12534-21ssh00:07:250
339930,0migration/330743-21unixbench-2d19:23:073
29715030,0irq/26-eth0-rx-0-21swapper/321:02:513
29715030,0irq/26-eth0-rx-0-21swapper/320:37:513
29715030,0irq/26-eth0-rx-0-21swapper/319:57:453
29715030,0irq/26-eth0-rx-0-21swapper/319:42:513
29715030,0irq/26-eth0-rx-0-21swapper/319:36:253
29715030,0irq/26-eth0-rx-0-21swapper/319:12:563
249930,0watchdog/20-21swapper/219:22:592
19130,0ktimersoftd/10-21swapper/120:38:281
179930,0migration/112788-21irqrtprio19:12:561
179930,0migration/111623-21kthreadcore19:57:581
169930,0watchdog/15441-21fschecks_time19:07:551
169930,0watchdog/12828-21diskmemload21:58:461
169930,0watchdog/124637-21kthreadcore00:12:571
169930,0watchdog/10-21swapper/120:17:501
169930,0watchdog/10-21swapper/120:02:581
46479921,0cyclictest32531-21aten_r1power_cu19:52:511
46479920,0cyclictest19119-21date19:42:401
46469921,0cyclictest944-21if_err_eth119:52:540
46469920,0cyclictest8877-21munin-node20:47:500
46469920,0cyclictest0-21swapper/020:12:480
46469920,0cyclictest0-21swapper/000:27:400
339920,0migration/327251-21diskstats20:12:533
29715020,0irq/26-eth0-rx-0-21swapper/320:57:563
29715020,0irq/26-eth0-rx-0-21swapper/320:32:513
29715020,0irq/26-eth0-rx-0-21swapper/319:37:543
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional