You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 04:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Apr 21, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1393223827,0sleep20-21swapper/218:57:362
304450350,0irq/25-eth10-21swapper/318:52:553
1376523424,0sleep10-21swapper/118:55:381
1387622917,0sleep00-21swapper/018:56:540
35160,0ktimersoftd/30-21swapper/323:35:023
339960,0migration/328280-21sh21:46:593
339960,0migration/32294-21sh21:28:133
339960,0migration/319675-21sh22:42:303
339960,0migration/318985-21sh21:58:553
339960,0migration/318676-21sh23:41:423
339960,0migration/317909-21sh23:18:423
339960,0migration/317755-21sh23:59:313
339960,0migration/311546-21sh22:37:163
339960,0migration/311156-21sh21:54:233
29725060,0irq/27-eth0-tx-30446-21munin-node20:23:150
29715060,0irq/26-eth0-rx-22198-21diskstats19:28:073
29715060,0irq/26-eth0-rx-21021-21kthreadcore00:03:073
140349960,0cyclictest0-21swapper/221:01:492
149960,0migration/026223-21sh00:05:540
149960,0migration/021932-21idleruntime21:23:060
149960,0migration/018203-21sh23:40:220
8215050,0irq/24-0000:00:12209-21latency_hist20:57:522
8215050,0irq/24-0000:00:0-21swapper/219:52:532
4150,0ktimersoftd/00-21swapper/022:33:210
4150,0ktimersoftd/00-21swapper/000:11:530
339950,0migration/34392-21fschecks_time22:53:053
339950,0migration/33797-21cpuspeed_turbos23:13:053
339950,0migration/330542-21aten_r1power_fr20:23:043
339950,0migration/329475-21latency_hist19:57:533
339950,0migration/328840-21users19:53:173
339950,0migration/328832-21gltestperf22:48:063
339950,0migration/327326-21taskset22:05:023
339950,0migration/326561-21taskset22:44:133
339950,0migration/326532-21taskset19:03:143
339950,0migration/326150-21switchtime22:23:173
339950,0migration/326057-21missed_timers21:03:133
339950,0migration/321790-21kthreadcore23:03:093
339950,0migration/321498-21kthreadcore00:23:093
339950,0migration/318268-21irqcore20:13:083
339950,0migration/318185-21kthreadcore21:38:103
339950,0migration/314236-21kthreadcore22:18:093
339950,0migration/312352-21kthreadcore20:08:103
29725050,0irq/27-eth0-tx-4362-21cut23:33:072
29725050,0irq/27-eth0-tx-4237-21ssh21:31:592
29725050,0irq/27-eth0-tx-3007-21munin-node23:53:132
29725050,0irq/27-eth0-tx-28471-21munin-node19:33:040
29725050,0irq/27-eth0-tx-27669-21munin-node23:28:062
29725050,0irq/27-eth0-tx-27088-21timerwakeupswit21:23:152
29725050,0irq/27-eth0-tx-12410-21hwlatdetect23:18:072
29725050,0irq/27-eth0-tx-10033-21memory21:53:112
29725050,0irq/27-eth0-tx-0-21swapper/223:46:012
29725050,0irq/27-eth0-tx-0-21swapper/223:24:472
29725050,0irq/27-eth0-tx-0-21swapper/222:48:292
29725050,0irq/27-eth0-tx-0-21swapper/222:03:032
29725050,0irq/27-eth0-tx-0-21swapper/221:42:022
29725050,0irq/27-eth0-tx-0-21swapper/200:04:182
29725050,0irq/27-eth0-tx-0-21swapper/020:53:040
29725050,0irq/27-eth0-tx-0-21swapper/020:18:110
29715050,0irq/26-eth0-rx-4994-21munin-node22:13:073
29715050,0irq/26-eth0-rx-3617-21sh22:11:483
29715050,0irq/26-eth0-rx-3387-21ssh21:10:263
29715050,0irq/26-eth0-rx-2495-21ssh22:29:233
29715050,0irq/26-eth0-rx-23926-21tr23:43:083
29715050,0irq/26-eth0-rx-21632-21ntpq19:48:133
29715050,0irq/26-eth0-rx-20487-21taskset19:01:003
29715050,0irq/26-eth0-rx-16451-21ssh00:18:123
29715050,0irq/26-eth0-rx-15436-21kthreadcore22:58:093
29715050,0irq/26-eth0-rx-0-21swapper/323:53:453
29715050,0irq/26-eth0-rx-0-21swapper/323:48:063
29715050,0irq/26-eth0-rx-0-21swapper/323:08:123
29715050,0irq/26-eth0-rx-0-21swapper/321:34:483
29715050,0irq/26-eth0-rx-0-21swapper/321:23:593
29715050,0irq/26-eth0-rx-0-21swapper/321:19:343
29715050,0irq/26-eth0-rx-0-21swapper/320:58:083
29715050,0irq/26-eth0-rx-0-21swapper/300:13:073
259950,0migration/24106-21sh21:12:182
259950,0migration/231438-21kthreadcore23:48:112
259950,0migration/230115-21kthreadcore23:08:092
259950,0migration/229410-21latency_hist19:57:532
259950,0migration/226685-21sh22:23:542
259950,0migration/226238-21kthreadcore20:43:082
259950,0migration/212371-21fschecks_time22:58:052
259950,0migration/212265-21kthreadcore20:08:102
259950,0migration/211075-21sh21:33:272
249950,0watchdog/25794-21kthreadcore22:33:062
249950,0watchdog/211941-21perl23:38:062
179950,0migration/17904-21kthreadcore23:33:091
179950,0migration/1325-21kthreadcore23:08:101
179950,0migration/131073-21kthreadcore22:08:101
179950,0migration/124264-21missed_timers00:23:111
179950,0migration/123514-21kthreadcore21:03:111
179950,0migration/121445-21latency19:48:111
179950,0migration/121303-21if_eth1.222:23:101
140349954,0cyclictest0-21swapper/221:13:112
140349950,0cyclictest0-21swapper/223:14:442
140349950,0cyclictest0-21swapper/221:21:322
140349950,0cyclictest0-21swapper/200:19:032
140329955,0cyclictest0-21swapper/021:45:340
140329954,0cyclictest13161-21awk22:18:070
140329950,0cyclictest0-21swapper/022:54:280
140329950,0cyclictest0-21swapper/021:38:480
149950,0migration/0978-21kthreadcore20:48:090
149950,0migration/08289-21open_files19:38:140
149950,0migration/05710-21kthreadcore23:33:080
149950,0migration/05140-21fschecks_count21:13:060
149950,0migration/04855-21aten_r1power_cu21:33:030
149950,0migration/03363-21runrttasks21:29:320
149950,0migration/032234-21kthreadcore19:58:100
149950,0migration/031525-21kthreadcore23:48:110
149950,0migration/028269-21sh22:07:460
149950,0migration/026876-21sh23:05:490
149950,0migration/025819-21needreboot22:23:130
149950,0migration/023244-21kthreadcore19:28:110
149950,0migration/021490-21fschecks_time19:03:050
149950,0migration/020867-21hddtemp_smartct22:43:060
149950,0migration/020788-21kthreadcore20:13:100
149950,0migration/02022-21run-parts22:48:520
149950,0migration/017330-21kthreadcore22:38:100
149950,0migration/013824-21kthreadcore00:18:080
149950,0migration/013548-21irqcore20:58:080
149950,0migration/012993-21kthreadcore20:33:090
149950,0migration/011036-21kthreadcore19:43:090
149950,0migration/010643-21kthreadcore19:18:090
149950,0migration/010107-21needreboot21:53:110
8215040,0irq/24-0000:00:10465-21sensors20:28:132
8215040,0irq/24-0000:00:0-21swapper/220:37:522
339940,0migration/34493-21latency_hist21:12:523
339940,0migration/327100-21kthreadcore20:18:123
339940,0migration/32426-21taskset19:37:203
29725040,0irq/27-eth0-tx-12643-21munin-node21:58:052
29725040,0irq/27-eth0-tx-12048-21ssh22:38:032
29725040,0irq/27-eth0-tx-0-21swapper/223:03:032
29725040,0irq/27-eth0-tx-0-21swapper/222:43:162
29725040,0irq/27-eth0-tx-0-21swapper/222:28:502
29725040,0irq/27-eth0-tx-0-21swapper/222:18:012
29725040,0irq/27-eth0-tx-0-21swapper/222:13:002
29725040,0irq/27-eth0-tx-0-21swapper/221:03:142
29725040,0irq/27-eth0-tx-0-21swapper/200:22:572
29725040,0irq/27-eth0-tx-0-21swapper/019:23:060
29715040,0irq/26-eth0-rx-0-21swapper/323:29:143
29715040,0irq/26-eth0-rx-0-21swapper/323:24:353
29715040,0irq/26-eth0-rx-0-21swapper/321:48:223
29715040,0irq/26-eth0-rx-0-21swapper/320:43:083
29715040,0irq/26-eth0-rx-0-21swapper/320:33:103
29715040,0irq/26-eth0-rx-0-21swapper/319:43:153
29715040,0irq/26-eth0-rx-0-21swapper/300:08:363
259940,0migration/27051-21kthreadcore22:53:092
259940,0migration/231526-21kthreadcore21:48:092
259940,0migration/231215-21hddtemp_smartct20:23:072
259940,0migration/226566-21kthreadcore19:28:122
259940,0migration/22410-21fschecks_time19:13:072
259940,0migration/223302-21fschecks_count19:53:072
259940,0migration/222058-21munin-node19:03:082
259940,0migration/220807-21latency_hist21:42:522
259940,0migration/21243-21kthreadcore20:48:092
249940,0watchdog/20-21swapper/223:58:582
249940,0watchdog/20-21swapper/222:08:062
249940,0watchdog/20-21swapper/220:03:072
249940,0watchdog/20-21swapper/200:08:102
179940,0migration/19989-21fschecks_count19:43:061
179940,0migration/17863-21kthreadcore22:53:091
179940,0migration/14240-21sh22:12:581
179940,0migration/13165-21runrttasks21:29:001
179940,0migration/130006-21latency20:43:101
179940,0migration/129502-21kthreadcore19:08:091
179940,0migration/128281-21latency_hist19:32:531
179940,0migration/128258-21sh22:48:041
179940,0migration/126442-21runrttasks23:46:001
179940,0migration/12353-21fschecks_time19:13:071
179940,0migration/123131-21kthreadcore22:43:091
179940,0migration/121905-21irqrtprio22:03:081
179940,0migration/116329-21timerwakeupswit20:08:161
179940,0migration/115087-21kthreadcore23:38:111
179940,0migration/115048-21kthreadcore22:38:091
179940,0migration/114007-21kthreadcore22:18:091
179940,0migration/111958-21irqcore00:18:071
179940,0migration/111097-21taskset23:58:011
169940,0watchdog/14780-21telnet20:28:031
169940,0watchdog/13007-21munin-node23:53:061
169940,0watchdog/10-21swapper/122:33:061
169940,0watchdog/10-21swapper/121:18:111
169940,0watchdog/10-21swapper/121:13:111
169940,0watchdog/10-21swapper/100:08:101
140349943,0cyclictest3556-21lldpd19:19:152
140349943,0cyclictest3514-21snmpd20:42:472
140349943,0cyclictest3514-21snmpd19:38:022
140349943,0cyclictest3514-21snmpd19:23:012
140349943,0cyclictest3514-21snmpd19:08:012
140349943,0cyclictest3514-21snmpd19:01:162
140349943,0cyclictest3514-21snmpd00:13:032
140349943,0cyclictest29401-21awk19:33:072
140349943,0cyclictest23846-21cut20:18:062
140349943,0cyclictest18015-21awk20:13:072
140349943,0cyclictest10323-21awk19:43:072
140339944,0cyclictest0-21swapper/122:59:221
140339944,0cyclictest0-21swapper/121:40:041
140339943,0cyclictest9421-21awk19:18:071
140339943,0cyclictest3556-21lldpd21:33:151
140339943,0cyclictest3556-21lldpd20:14:151
140339943,0cyclictest3556-21lldpd20:03:461
140339943,0cyclictest3514-21snmpd23:03:481
140339943,0cyclictest3241-21irqbalance19:59:311
140339943,0cyclictest3241-21irqbalance19:54:311
140339943,0cyclictest32244-21awk20:48:071
140339943,0cyclictest31426-21awk20:23:081
140339943,0cyclictest29751-21awk21:48:071
140339943,0cyclictest28597-21sadc21:47:521
140339943,0cyclictest28350-21awk23:48:071
140339943,0cyclictest24731-21awk20:18:101
140339943,0cyclictest22002-21awk21:23:071
140339943,0cyclictest21795-21awk19:03:071
140339940,0cyclictest0-21swapper/123:32:381
140339940,0cyclictest0-21swapper/123:27:161
140339940,0cyclictest0-21swapper/100:13:181
140329944,0cyclictest0-21swapper/023:25:320
140329944,0cyclictest0-21swapper/022:13:100
140329943,0cyclictest4661-21awk20:03:070
140329943,0cyclictest3514-21snmpd23:16:480
140329943,0cyclictest3514-21snmpd22:59:030
140329943,0cyclictest29754-21awk21:08:080
140329943,0cyclictest29256-21awk22:28:080
140329943,0cyclictest19753-21awk00:23:070
140329943,0cyclictest19345-21crond23:22:520
140329943,0cyclictest17040-21awk19:48:080
140329943,0cyclictest13010-21kworker/0:221:03:140
149940,0migration/09614-21sh00:15:060
149940,0migration/029999-21latency20:43:100
149940,0migration/029303-21forks21:48:050
149940,0migration/019232-21latency_hist23:42:520
149940,0migration/018146-21sh00:00:400
149940,0migration/014535-21cpuspeed_turbos18:58:050
149940,0migration/012275-21kthreadcore20:08:100
35130,0ktimersoftd/30-21swapper/320:31:313
339930,0migration/320869-21kthreadcore20:38:073
29715030,0irq/26-eth0-rx-0-21swapper/320:55:323
29715030,0irq/26-eth0-rx-0-21swapper/319:38:063
29715030,0irq/26-eth0-rx-0-21swapper/319:23:023
29715030,0irq/26-eth0-rx-0-21swapper/319:22:343
29715030,0irq/26-eth0-rx-0-21swapper/319:08:043
179930,0migration/115684-21fschecks_count19:23:061
169930,0watchdog/10-21swapper/123:17:581
169930,0watchdog/10-21swapper/122:28:061
169930,0watchdog/10-21swapper/121:55:311
169930,0watchdog/10-21swapper/121:10:071
169930,0watchdog/10-21swapper/118:57:591
140339933,0cyclictest3556-21lldpd20:41:161
140339932,0cyclictest4458-21awk23:13:081
140339932,0cyclictest3514-21snmpd20:58:181
140339932,0cyclictest29323-21awk19:33:061
140339932,0cyclictest24952-21sensors00:03:121
140339932,0cyclictest14846-21grep21:58:091
140329933,0cyclictest13010-21kworker/0:223:53:190
140329933,0cyclictest13010-21kworker/0:223:27:550
140329933,0cyclictest13010-21kworker/0:223:07:590
140329933,0cyclictest13010-21kworker/0:222:08:000
140329933,0cyclictest13010-21kworker/0:220:37:540
140329933,0cyclictest13010-21kworker/0:220:28:000
140329933,0cyclictest13010-21kworker/0:219:52:580
140329933,0cyclictest13010-21kworker/0:219:12:580
140329933,0cyclictest13010-21kworker/0:219:07:560
140329932,0cyclictest3514-21snmpd21:17:530
140329932,0cyclictest14514-21kthreadcore21:58:090
339920,0migration/332411-21iostat_ios20:48:083
29715020,0irq/26-eth0-rx-0-21swapper/320:03:043
29715020,0irq/26-eth0-rx-0-21swapper/319:13:103
140339920,0cyclictest11423-21latency_hist20:32:521
140339920,0cyclictest0-21swapper/120:52:521
140339920,0cyclictest0-21swapper/119:37:531
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional