You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-08 - 04:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Oct 08, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450850,0irq/25-eth10-21swapper/318:40:483
2259823824,0sleep00-21swapper/018:39:340
2240023525,0sleep10-21swapper/118:37:151
2270123324,0sleep20-21swapper/218:40:482
339960,0migration/3704-21sh23:08:413
339960,0migration/33316-21unixbench_singl18:47:063
339960,0migration/327880-21sh21:02:443
339960,0migration/310875-21sh21:53:503
29725060,0irq/27-eth0-tx-27023-21munin-node20:22:020
29715060,0irq/26-eth0-rx-4370-21ssh21:31:343
29715060,0irq/26-eth0-rx-26677-21users21:42:043
29715060,0irq/26-eth0-rx-24624-21grep22:01:593
228559960,0cyclictest0-21swapper/223:00:422
228539960,0cyclictest3576-21ssh20:48:301
228539960,0cyclictest0-21swapper/122:35:141
228539960,0cyclictest0-21swapper/100:08:431
228529965,0cyclictest0-21swapper/020:36:570
149960,0migration/09938-21sh22:34:010
149960,0migration/02672-21sh22:31:230
149960,0migration/026104-21sh22:23:220
149960,0migration/016378-21sh00:00:300
149960,0migration/01034-21sh23:31:070
4150,0ktimersoftd/00-21swapper/020:41:520
35150,0ktimersoftd/30-21swapper/323:41:523
35150,0ktimersoftd/30-21swapper/320:58:513
35150,0ktimersoftd/30-21swapper/320:41:523
339950,0migration/39026-21taskset22:52:413
339950,0migration/36942-21kthreadcore22:31:593
339950,0migration/36205-21munin-node21:11:553
339950,0migration/3431-21kthreadcore20:46:593
339950,0migration/330524-21sh00:07:003
339950,0migration/329797-21munin-node21:46:573
339950,0migration/329280-21taskset18:44:263
339950,0migration/328113-21kthreadcore23:26:563
339950,0migration/325834-21munin-node23:46:533
339950,0migration/325228-21sh23:25:153
339950,0migration/324441-21munin-node19:06:553
339950,0migration/322489-21kthreadcore22:21:573
339950,0migration/321372-21kthreadcore23:02:013
339950,0migration/319937-21munin-node20:36:583
339950,0migration/318853-21fschecks_count19:26:543
339950,0migration/318853-21fschecks_count19:26:543
339950,0migration/314969-21kthreadcore21:56:573
339950,0migration/312747-21kthreadcore23:16:593
339950,0migration/312295-21kthreadcore22:56:573
339950,0migration/312184-21taskset21:36:003
339950,0migration/311504-21sh18:56:573
339950,0migration/310418-21munin-node23:36:583
29725050,0irq/27-eth0-tx-3317-21vmstat21:07:042
29725050,0irq/27-eth0-tx-0-21swapper/020:32:050
29725050,0irq/27-eth0-tx-0-21swapper/020:12:050
29725050,0irq/27-eth0-tx-0-21swapper/019:37:270
29715050,0irq/26-eth0-rx-897-21switchtime22:27:043
29715050,0irq/26-eth0-rx-8661-21ssh23:34:473
29715050,0irq/26-eth0-rx-8290-21kthreadcore19:41:593
29715050,0irq/26-eth0-rx-6559-21runrttasks22:11:513
29715050,0irq/26-eth0-rx-5945-21kthreadcore23:11:583
29715050,0irq/26-eth0-rx-0-21swapper/323:52:513
29715050,0irq/26-eth0-rx-0-21swapper/322:36:433
29715050,0irq/26-eth0-rx-0-21swapper/322:16:593
29715050,0irq/26-eth0-rx-0-21swapper/321:36:573
29715050,0irq/26-eth0-rx-0-21swapper/321:16:553
29715050,0irq/26-eth0-rx-0-21swapper/320:51:473
228559955,0cyclictest24533-21timerwakeupswit23:02:072
228559955,0cyclictest0-21swapper/223:15:592
228559955,0cyclictest0-21swapper/221:49:262
228559950,0cyclictest0-21swapper/223:54:462
228559950,0cyclictest0-21swapper/223:45:282
228559950,0cyclictest0-21swapper/223:27:212
228559950,0cyclictest0-21swapper/221:38:152
228559950,0cyclictest0-21swapper/220:55:032
228559950,0cyclictest0-21swapper/220:42:162
228539955,0cyclictest0-21swapper/122:43:071
228539955,0cyclictest0-21swapper/121:31:511
228539950,0cyclictest0-21swapper/122:39:101
228529955,0cyclictest0-21swapper/022:16:540
228529954,0cyclictest0-21swapper/021:46:420
228529950,0cyclictest0-21swapper/023:24:130
228529950,0cyclictest0-21swapper/000:05:510
179950,0migration/15017-21munin-node21:51:561
179950,0migration/130434-21gltestperf18:46:561
179950,0migration/130030-21kthreadcore23:26:571
179950,0migration/128457-21taskset21:25:411
179950,0migration/121517-21kthreadcore22:21:571
179950,0migration/117102-21kthreadcore20:36:571
179950,0migration/116034-21missed_timers22:56:591
179950,0migration/114887-21munin-node20:11:591
169950,0watchdog/128759-21ls18:42:051
169950,0watchdog/114748-21grep18:57:011
149950,0migration/05105-21munin-node21:51:570
149950,0migration/04123-21kthreadcore20:26:570
149950,0migration/031269-21sh00:07:330
149950,0migration/031074-21kthreadcore22:46:590
149950,0migration/028123-21kthreadcore19:31:590
149950,0migration/025887-21latency_hist19:56:430
149950,0migration/024564-21fschecks_count19:06:560
149950,0migration/02358-21kthreadcore18:47:000
149950,0migration/021662-21fschecks_count21:21:540
149950,0migration/020881-21sh21:01:010
149950,0migration/018782-21cpuspeed_turbos23:01:560
149950,0migration/018363-21munin-node19:01:580
149950,0migration/01780-21fschecks_time23:51:560
149950,0migration/017094-21kthreadcore19:46:570
149950,0migration/015383-21kthreadcore22:56:580
149950,0migration/013355-21irqrtprio21:56:570
149950,0migration/011446-21taskset21:12:150
149950,0migration/010068-21kthreadcore19:17:000
8215040,0irq/24-0000:00:6911-21cut21:31:582
339940,0migration/35643-21kthreadcore18:51:563
339940,0migration/322553-21kthreadcore20:16:583
339940,0migration/319614-21fschecks_count22:41:553
339940,0migration/31450-21kthreadcore19:12:003
339940,0migration/312348-21fschecks_time19:21:553
339940,0migration/310341-21kthreadcore20:31:593
29725040,0irq/27-eth0-tx-5159-21kthreadcore22:11:562
29725040,0irq/27-eth0-tx-21087-21diskmemload21:58:312
29725040,0irq/27-eth0-tx-16418-21grep22:17:002
29725040,0irq/27-eth0-tx-0-21swapper/223:37:002
29725040,0irq/27-eth0-tx-0-21swapper/223:23:102
29725040,0irq/27-eth0-tx-0-21swapper/222:26:582
29725040,0irq/27-eth0-tx-0-21swapper/221:51:592
29725040,0irq/27-eth0-tx-0-21swapper/200:06:522
29715040,0irq/26-eth0-rx-20185-21kthreadcore00:02:003
29715040,0irq/26-eth0-rx-0-21swapper/323:57:243
29715040,0irq/26-eth0-rx-0-21swapper/322:46:543
29715040,0irq/26-eth0-rx-0-21swapper/322:10:523
29715040,0irq/26-eth0-rx-0-21swapper/321:26:143
29715040,0irq/26-eth0-rx-0-21swapper/321:07:493
259940,0migration/28648-21kthreadcore19:16:592
259940,0migration/23316-21kthreadcore23:31:572
259940,0migration/230179-21kthreadcore23:46:582
259940,0migration/227295-21cpu22:46:542
259940,0migration/224097-21irqcore18:41:592
259940,0migration/222768-21kthreadcore22:01:592
249940,0watchdog/28138-21kthreadcore21:11:562
249940,0watchdog/21368-21ssh23:10:322
249940,0watchdog/213502-21cpu20:56:522
249940,0watchdog/20-21swapper/222:51:522
249940,0watchdog/20-21swapper/220:06:572
249940,0watchdog/20-21swapper/219:47:012
249940,0watchdog/20-21swapper/219:21:532
228559944,0cyclictest0-21swapper/222:37:122
228559944,0cyclictest0-21swapper/222:08:402
228559944,0cyclictest0-21swapper/221:18:512
228559943,0cyclictest4768-21sadc20:51:422
228559943,0cyclictest3556-21lldpd22:42:032
228559943,0cyclictest3556-21lldpd19:11:322
228559943,0cyclictest3514-21snmpd19:43:282
228559943,0cyclictest3514-21snmpd19:00:282
228559943,0cyclictest3241-21irqbalance21:42:402
228559943,0cyclictest3241-21irqbalance21:05:102
228559943,0cyclictest3241-21irqbalance18:52:012
228559943,0cyclictest3241-21irqbalance18:47:012
228559943,0cyclictest29926-21awk21:26:542
228559943,0cyclictest27978-21awk20:21:572
228559943,0cyclictest21540-21diskstats21:21:532
228559943,0cyclictest21292-21awk20:16:562
228559943,0cyclictest19302-21awk19:26:562
228559943,0cyclictest19302-21awk19:26:562
228559943,0cyclictest18271-21awk19:01:572
228559943,0cyclictest17963-21awk00:01:582
228559943,0cyclictest14498-21awk20:11:572
228559943,0cyclictest0-21swapper/223:17:002
228539944,0cyclictest0-21swapper/123:59:441
228539944,0cyclictest0-21swapper/121:36:541
228539943,0cyclictest4518-21awk22:11:551
228539943,0cyclictest3514-21snmpd23:01:441
228539943,0cyclictest3514-21snmpd22:16:441
228539943,0cyclictest3514-21snmpd20:56:591
228539943,0cyclictest3514-21snmpd20:46:291
228539943,0cyclictest3514-21snmpd19:40:281
228539943,0cyclictest3514-21snmpd19:11:281
228539943,0cyclictest3514-21snmpd19:06:281
228539943,0cyclictest27860-21awk20:21:571
228539943,0cyclictest21030-21awk22:01:571
228539943,0cyclictest19214-21awk19:26:551
228539943,0cyclictest19214-21awk19:26:551
228539943,0cyclictest18962-21awk23:21:561
228539943,0cyclictest13516-21awk19:46:541
228529944,0cyclictest0-21swapper/022:11:330
228529944,0cyclictest0-21swapper/021:50:250
228529944,0cyclictest0-21swapper/021:32:170
228529943,0cyclictest3514-21snmpd23:20:290
228529943,0cyclictest3514-21snmpd20:01:590
228529943,0cyclictest30057-21awk21:26:550
228529943,0cyclictest23901-21awk18:41:570
228529943,0cyclictest20016-21awk22:41:570
228529943,0cyclictest14149-21awk21:16:570
228529943,0cyclictest12523-21awk19:21:560
228529943,0cyclictest11571-21awk18:56:580
228529942,0cyclictest26869-21kworker/0:123:37:030
179940,0migration/1948-21latency_hist23:51:411
179940,0migration/18434-21kthreadcore19:16:591
179940,0migration/18209-21cpu20:31:551
179940,0migration/17395-21kthreadcore21:11:561
179940,0migration/17242-21kthreadcore20:51:591
179940,0migration/14623-21taskset19:15:071
179940,0migration/13440-21kthreadcore23:11:571
179940,0migration/131332-21kthreadcore21:06:571
179940,0migration/128700-21kthreadcore19:56:571
179940,0migration/128058-21fschecks_time22:26:551
179940,0migration/127334-21cpuspeed_turbos22:46:541
179940,0migration/127068-21ksmtuned21:43:111
179940,0migration/126117-21kthreadcore21:01:591
179940,0migration/121022-21fschecks_time20:16:551
179940,0migration/119293-21taskset19:51:451
179940,0migration/118201-21irqcore00:01:591
179940,0migration/11750-21fschecks_time23:51:561
179940,0migration/113074-21kthreadcore23:37:011
179940,0migration/111359-21irqrtprio23:16:581
179940,0migration/110608-21kthreadcore19:42:001
169940,0watchdog/18175-21netstat22:52:001
169940,0watchdog/128487-21ssh21:46:481
169940,0watchdog/123946-21ssh23:42:521
169940,0watchdog/117149-21munin-node19:22:011
169940,0watchdog/110024-21cut20:06:571
169940,0watchdog/10-21swapper/123:31:561
169940,0watchdog/10-21swapper/123:11:361
169940,0watchdog/10-21swapper/119:31:531
169940,0watchdog/10-21swapper/118:51:571
149940,0migration/03077-21iostat23:11:560
149940,0migration/025508-21aten_r1power_cu23:46:510
149940,0migration/018324-21sh22:40:180
149940,0migration/015640-21kthreadcore21:36:570
149940,0migration/011348-21kthreadcore20:06:570
8215030,0irq/24-0000:00:0-21swapper/219:56:542
29725030,0irq/27-eth0-tx-0-21swapper/222:21:452
29715030,0irq/26-eth0-rx-0-21swapper/320:21:543
29715030,0irq/26-eth0-rx-0-21swapper/320:11:573
29715030,0irq/26-eth0-rx-0-21swapper/320:11:343
29715030,0irq/26-eth0-rx-0-21swapper/320:01:583
29715030,0irq/26-eth0-rx-0-21swapper/319:58:113
29715030,0irq/26-eth0-rx-0-21swapper/319:46:513
29715030,0irq/26-eth0-rx-0-21swapper/319:36:533
29715030,0irq/26-eth0-rx-0-21swapper/319:17:013
29715030,0irq/26-eth0-rx-0-21swapper/319:01:543
259930,0migration/24712-21taskset19:37:002
249930,0watchdog/20-21swapper/220:02:012
249930,0watchdog/20-21swapper/219:51:532
249930,0watchdog/20-21swapper/219:31:452
228559932,0cyclictest3514-21snmpd20:31:442
228559932,0cyclictest3514-21snmpd20:29:442
228559932,0cyclictest31033-21sed19:11:542
228559932,0cyclictest3059-21munin-node22:31:532
228559932,0cyclictest15532-21awk20:36:552
228559932,0cyclictest10064-21awk23:56:552
228539933,0cyclictest21087-21diskmemload22:10:571
228539932,0cyclictest2644-21grep20:26:571
228539930,0cyclictest12259-21aten2.4-expect21:56:521
228529933,0cyclictest26869-21kworker/0:123:41:470
228529933,0cyclictest26869-21kworker/0:123:31:430
228529933,0cyclictest26869-21kworker/0:123:06:430
228529933,0cyclictest26869-21kworker/0:122:51:430
228529933,0cyclictest26869-21kworker/0:122:11:490
228529933,0cyclictest26869-21kworker/0:122:01:590
228529933,0cyclictest26869-21kworker/0:121:06:510
228529933,0cyclictest26869-21kworker/0:121:01:490
228529933,0cyclictest26869-21kworker/0:120:51:440
228529933,0cyclictest26869-21kworker/0:120:46:440
228529933,0cyclictest26869-21kworker/0:119:41:500
228529933,0cyclictest26869-21kworker/0:119:26:480
228529933,0cyclictest26869-21kworker/0:119:26:480
228529933,0cyclictest26869-21kworker/0:119:11:500
228529933,0cyclictest26869-21kworker/0:118:51:500
228529932,0cyclictest19304-21munin-node19:51:550
228529930,0cyclictest24910-21grep20:16:580
179930,0migration/14396-21kthreadcore20:02:001
179930,0migration/129193-21aten_r1power_vo21:26:511
169930,0watchdog/10-21swapper/121:17:361
329920,0watchdog/34811-21kthreadcore20:26:573
29715020,0irq/26-eth0-rx-0-21swapper/319:51:503
29715020,0irq/26-eth0-rx-0-21swapper/319:34:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional