You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-16 - 18:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sun Nov 16, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1207826766,0sleep20-21swapper/206:35:572
291350560,0irq/25-eth10-21swapper/306:36:063
1219824534,0sleep00-21swapper/006:37:230
1209323026,0sleep10-21swapper/106:36:081
28415070,0irq/27-eth0-tx-0-21swapper/210:21:562
28415070,0irq/27-eth0-tx-0-21swapper/008:41:180
28415070,0irq/27-eth0-tx-0-21swapper/008:25:020
249970,0watchdog/20-21swapper/211:10:072
249970,0watchdog/20-21swapper/209:46:562
124839970,0cyclictest16799-21ssh09:47:181
339960,0migration/37367-21cpuspeed_turbos07:00:033
339960,0migration/328646-21timerwakeupswit10:55:163
339960,0migration/325003-21sh09:53:203
339960,0migration/322789-21latency08:25:083
339960,0migration/310949-21kthreadcore07:50:073
28415060,0irq/27-eth0-tx-32736-21munin-node06:55:120
28415060,0irq/27-eth0-tx-23819-21ssh10:13:002
28415060,0irq/27-eth0-tx-0-21swapper/212:06:262
28415060,0irq/27-eth0-tx-0-21swapper/212:00:402
28415060,0irq/27-eth0-tx-0-21swapper/211:50:322
28415060,0irq/27-eth0-tx-0-21swapper/211:46:332
28415060,0irq/27-eth0-tx-0-21swapper/211:42:342
28415060,0irq/27-eth0-tx-0-21swapper/211:29:272
28415060,0irq/27-eth0-tx-0-21swapper/211:06:142
28415060,0irq/27-eth0-tx-0-21swapper/211:03:282
28415060,0irq/27-eth0-tx-0-21swapper/210:56:282
28415060,0irq/27-eth0-tx-0-21swapper/210:54:212
28415060,0irq/27-eth0-tx-0-21swapper/210:35:082
28415060,0irq/27-eth0-tx-0-21swapper/210:34:392
28415060,0irq/27-eth0-tx-0-21swapper/210:19:022
28415060,0irq/27-eth0-tx-0-21swapper/210:01:152
28415060,0irq/27-eth0-tx-0-21swapper/209:28:542
28415060,0irq/27-eth0-tx-0-21swapper/209:16:432
28415060,0irq/27-eth0-tx-0-21swapper/209:09:112
28415060,0irq/27-eth0-tx-0-21swapper/209:04:402
28415060,0irq/27-eth0-tx-0-21swapper/208:56:152
28415060,0irq/27-eth0-tx-0-21swapper/208:50:042
28415060,0irq/27-eth0-tx-0-21swapper/208:42:172
28415060,0irq/27-eth0-tx-0-21swapper/008:20:100
28415060,0irq/27-eth0-tx-0-21swapper/007:52:570
28405060,0irq/26-eth0-rx-32351-21cut11:40:083
28405060,0irq/26-eth0-rx-29320-21cut10:15:083
28405060,0irq/26-eth0-rx-14144-21irqcore08:45:063
249960,0watchdog/220017-21munin-node09:30:082
149960,0migration/01755-21sh09:39:430
124829960,0cyclictest0-21swapper/010:40:140
4150,0ktimersoftd/00-21swapper/011:36:020
4150,0ktimersoftd/00-21swapper/011:19:580
4150,0ktimersoftd/00-21swapper/010:21:380
4150,0ktimersoftd/00-21swapper/008:53:300
339950,0migration/37825-21munin-node10:45:053
339950,0migration/37198-21kthreadcore11:45:083
339950,0migration/36909-21kthreadcore12:05:093
339950,0migration/36176-21if_eth108:40:063
339950,0migration/34022-21gltestperf09:20:043
339950,0migration/332663-21kthreadcore08:35:103
339950,0migration/332400-21taskset10:40:053
339950,0migration/331475-21munin-node09:55:143
339950,0migration/329851-21munin-node08:30:123
339950,0migration/327465-21kthreadcore06:50:063
339950,0migration/326540-21kthreadcore10:35:073
339950,0migration/32401-21cpuspeed_turbos09:40:043
339950,0migration/322955-21irqrtprio11:55:103
339950,0migration/32250-21latency_hist07:44:493
339950,0migration/320460-21aten_r1power_fr07:10:043
339950,0migration/319898-21fschecks_count06:45:143
339950,0migration/314234-21df_abs11:10:033
339950,0migration/313216-21kthreadcore09:45:103
339950,0migration/311477-21munin-node09:25:033
339950,0migration/311159-21kthreadcore10:25:083
28415050,0irq/27-eth0-tx-3900-21ssh11:21:082
28415050,0irq/27-eth0-tx-0-21swapper/211:56:352
28415050,0irq/27-eth0-tx-0-21swapper/211:35:482
28415050,0irq/27-eth0-tx-0-21swapper/211:14:532
28415050,0irq/27-eth0-tx-0-21swapper/210:44:502
28415050,0irq/27-eth0-tx-0-21swapper/209:58:032
28415050,0irq/27-eth0-tx-0-21swapper/209:42:522
28415050,0irq/27-eth0-tx-0-21swapper/209:20:372
28415050,0irq/27-eth0-tx-0-21swapper/209:12:562
28415050,0irq/27-eth0-tx-0-21swapper/008:00:080
28415050,0irq/27-eth0-tx-0-21swapper/007:10:170
28405050,0irq/26-eth0-rx-719-21taskset09:36:153
28405050,0irq/26-eth0-rx-28975-21irqrtprio09:15:063
28405050,0irq/26-eth0-rx-28750-21ssh11:38:043
28405050,0irq/26-eth0-rx-23460-21kthreadcore07:35:093
28405050,0irq/26-eth0-rx-21529-21kthreadcore09:10:083
28405050,0irq/26-eth0-rx-19643-21unixbench_multi11:50:133
28405050,0irq/26-eth0-rx-12451-21cut10:05:073
28405050,0irq/26-eth0-rx-0-21swapper/312:00:073
28405050,0irq/26-eth0-rx-0-21swapper/311:30:033
28405050,0irq/26-eth0-rx-0-21swapper/311:28:383
28405050,0irq/26-eth0-rx-0-21swapper/311:22:273
28405050,0irq/26-eth0-rx-0-21swapper/311:04:503
28405050,0irq/26-eth0-rx-0-21swapper/311:00:243
28405050,0irq/26-eth0-rx-0-21swapper/310:52:593
28405050,0irq/26-eth0-rx-0-21swapper/310:32:223
28405050,0irq/26-eth0-rx-0-21swapper/308:55:123
28405050,0irq/26-eth0-rx-0-21swapper/308:50:213
28405050,0irq/26-eth0-rx-0-21swapper/308:20:073
259950,0migration/24900-21gltestperf08:15:032
259950,0migration/229016-21munin-node07:40:062
259950,0migration/226734-21latency_hist09:34:492
259950,0migration/224287-21kthreadcore07:10:102
259950,0migration/223128-21kthreadcore07:35:092
259950,0migration/220139-21kthreadcore08:25:072
259950,0migration/216281-21kthreadcore11:30:072
259950,0migration/210497-21kthreadcore10:05:072
249950,0watchdog/224125-21ssh09:50:322
249950,0watchdog/20-21swapper/210:25:072
179950,0migration/14607-21fschecks_time09:00:061
179950,0migration/130101-21kthreadcore09:55:091
179950,0migration/121344-21latency_hist07:34:491
179950,0migration/120611-21cpuspeed_turbos07:10:051
179950,0migration/119871-21kthreadcore09:50:071
179950,0migration/116480-21kthreadcore10:50:061
179950,0migration/116364-21munin-node08:20:101
179950,0migration/110179-21kthreadcore10:25:081
169950,0watchdog/131744-21kthreadcore08:05:081
169950,0watchdog/125711-21taskset06:48:561
169950,0watchdog/12360-21cut09:40:041
169950,0watchdog/122007-21df_inode11:35:031
169950,0watchdog/112128-21ssh08:42:521
169950,0watchdog/111009-21kthreadcore07:25:081
169950,0watchdog/10-21swapper/111:46:591
169950,0watchdog/10-21swapper/111:43:151
169950,0watchdog/10-21swapper/111:28:591
169950,0watchdog/10-21swapper/111:00:031
169950,0watchdog/10-21swapper/109:23:561
169950,0watchdog/10-21swapper/107:44:201
149950,0migration/09326-21kthreadcore08:15:070
149950,0migration/09065-21kthreadcore11:45:080
149950,0migration/06305-21cpuspeed_turbos11:05:110
149950,0migration/05892-21kthreadcore07:45:060
149950,0migration/04666-21kthreadcore07:20:090
149950,0migration/032392-21kthreadcore09:15:080
149950,0migration/032116-21kthreadcore08:55:110
149950,0migration/030490-21taskset08:33:040
149950,0migration/030470-21latency_hist10:59:480
149950,0migration/030140-21taskset10:59:290
149950,0migration/029943-21kthreadcore10:15:080
149950,0migration/029509-21df_abs08:05:030
149950,0migration/024822-21kthreadcore07:35:100
149950,0migration/024436-21gltestperf10:35:040
149950,0migration/020566-21gltestperf09:10:050
149950,0migration/020249-21kthreadcore07:55:080
149950,0migration/018966-21taskset09:08:120
149950,0migration/016667-21idleruntime10:30:050
149950,0migration/014513-21kthreadcore09:25:080
149950,0migration/010637-21ksmtuned09:01:480
149950,0migration/010189-21aten_r1power_po09:45:030
124849950,0cyclictest0-21swapper/210:41:462
124839955,0cyclictest0-21swapper/110:37:551
124839955,0cyclictest0-21swapper/109:19:151
124839955,0cyclictest0-21swapper/109:09:191
124839955,0cyclictest0-21swapper/108:52:081
124839955,0cyclictest0-21swapper/107:14:511
124839950,0cyclictest0-21swapper/111:20:571
124839950,0cyclictest0-21swapper/109:26:111
124839950,0cyclictest0-21swapper/108:48:301
124829955,0cyclictest0-21swapper/011:41:430
124829955,0cyclictest0-21swapper/010:00:580
124829950,0cyclictest0-21swapper/012:07:280
124829950,0cyclictest0-21swapper/011:31:000
124829950,0cyclictest0-21swapper/011:28:190
124829950,0cyclictest0-21swapper/009:51:590
339940,0migration/39819-21kthreadcore07:25:073
339940,0migration/37798-21kthreadcore08:15:073
339940,0migration/35449-21kthreadcore06:55:083
339940,0migration/330923-21kthreadcore08:05:073
339940,0migration/324389-21kthreadcore11:15:073
339940,0migration/321648-21kthreadcore10:10:113
339940,0migration/316546-21kthreadcore06:40:073
28415040,0irq/27-eth0-tx-0-21swapper/007:05:040
28405040,0irq/26-eth0-rx-0-21swapper/310:00:033
28405040,0irq/26-eth0-rx-0-21swapper/309:30:203
28405040,0irq/26-eth0-rx-0-21swapper/309:05:523
28405040,0irq/26-eth0-rx-0-21swapper/309:00:043
28405040,0irq/26-eth0-rx-0-21swapper/307:31:563
259940,0migration/28869-21kthreadcore07:00:082
259940,0migration/227354-21kthreadcore08:30:082
259940,0migration/220894-21kthreadcore06:45:172
259940,0migration/21764-21kthreadcore08:35:102
259940,0migration/213500-21munin-node06:40:042
259940,0migration/212344-21kthreadcore08:20:072
19140,0ktimersoftd/17005-21munin-node10:45:141
179940,0migration/15979-21fschecks_count12:05:051
179940,0migration/128024-21kthreadcore10:55:101
179940,0migration/122230-21taskset10:31:031
179940,0migration/12176-21kthreadcore06:55:071
179940,0migration/12039-21gltestperf07:20:051
179940,0migration/11594-21kthreadcore08:35:101
179940,0migration/114875-21kernelversion11:50:061
179940,0migration/113802-21ksmtuned11:09:491
169940,0watchdog/126963-21kthreadcore08:30:081
169940,0watchdog/122791-21cpuspeed_turbos08:00:041
169940,0watchdog/118184-21fschecks_time08:25:041
169940,0watchdog/11818-21kthreadcore08:55:121
169940,0watchdog/10-21swapper/112:00:511
169940,0watchdog/10-21swapper/110:40:031
169940,0watchdog/10-21swapper/109:37:121
169940,0watchdog/10-21swapper/109:32:241
169940,0watchdog/10-21swapper/107:05:161
149940,0migration/030260-21kthreadcore07:15:080
149940,0migration/029593-21latency_hist11:19:480
149940,0migration/028709-21fschecks_time07:40:040
149940,0migration/025392-21kthreadcore11:55:110
149940,0migration/022133-21kthreadcore10:10:110
149940,0migration/019648-21fschecks_time09:30:050
149940,0migration/018447-21timerandwakeup06:40:120
149940,0migration/016099-21kthreadcore11:50:060
124849943,0cyclictest8957-21awk07:25:052
124849943,0cyclictest7821-21tail07:45:082
124849943,0cyclictest6445-21snmpd08:08:582
124849943,0cyclictest6445-21snmpd07:59:582
124849943,0cyclictest6445-21snmpd07:52:582
124849943,0cyclictest6445-21snmpd06:58:432
124849943,0cyclictest16723-21awk07:55:062
124849943,0cyclictest14609-21awk07:05:072
124849943,0cyclictest13967-21awk08:45:052
124839944,0cyclictest0-21swapper/110:19:571
124839944,0cyclictest0-21swapper/110:13:571
124839944,0cyclictest0-21swapper/107:38:491
124839943,0cyclictest9990-21awk07:50:051
124839943,0cyclictest9983-21awk10:05:051
124839943,0cyclictest30475-21kworker/1:111:56:391
124839943,0cyclictest22766-21awk11:15:051
124839943,0cyclictest2031-21awk10:00:071
124839943,0cyclictest14581-21kworker/1:211:31:471
124839943,0cyclictest14581-21kworker/1:210:15:031
124829944,0cyclictest0-21swapper/010:24:560
124829943,0cyclictest6445-21snmpd07:34:430
124829943,0cyclictest20270-21awk06:45:150
124829943,0cyclictest14725-21awk11:10:050
124829940,0cyclictest8121-21munin-node07:25:070
28405030,0irq/26-eth0-rx-0-21swapper/310:20:023
28405030,0irq/26-eth0-rx-0-21swapper/308:10:163
28405030,0irq/26-eth0-rx-0-21swapper/308:00:053
28405030,0irq/26-eth0-rx-0-21swapper/307:55:133
28405030,0irq/26-eth0-rx-0-21swapper/307:16:293
28405030,0irq/26-eth0-rx-0-21swapper/307:05:043
249930,0watchdog/20-21swapper/207:15:042
179930,0migration/120380-21kthreadcore07:55:081
169930,0watchdog/10-21swapper/108:10:481
169930,0watchdog/10-21swapper/107:00:081
169930,0watchdog/10-21swapper/106:50:001
149930,0migration/014821-21sh10:49:520
124849932,0cyclictest26766-21grep06:50:032
124849932,0cyclictest2256-21awk07:20:072
124849930,0cyclictest0-21swapper/207:34:492
124839933,0cyclictest14581-21kworker/1:211:05:051
124839933,0cyclictest14581-21kworker/1:209:09:571
124839933,0cyclictest14581-21kworker/1:207:44:511
124839933,0cyclictest14581-21kworker/1:206:40:091
124839930,0cyclictest20-21ksoftirqd/108:15:071
124829933,0cyclictest3425-21lldpd09:20:040
124829933,0cyclictest0-21swapper/012:01:060
124829932,0cyclictest6445-21snmpd10:48:290
124829932,0cyclictest6445-21snmpd08:44:580
124829932,0cyclictest2869-21sed09:40:050
124829932,0cyclictest12750-21python07:00:130
124829931,0cyclictest3662-21kthreadcore08:35:110
124829930,0cyclictest28086-21cut06:50:060
124829930,0cyclictest0-21swapper/008:12:020
8225020,0irq/24-0000:00:0-21swapper/208:09:492
339920,0migration/31718-21diskstats07:20:043
28405020,0irq/26-eth0-rx-0-21swapper/307:45:003
124829920,0cyclictest0-21swapper/010:04:520
124829920,0cyclictest0-21swapper/009:54:510
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional