You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 01:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Dec 07, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350920,0irq/25-eth10-21swapper/318:37:473
271850,0ktimersoftd/222553-21perf18:38:542
2888823531,0sleep00-21swapper/018:36:090
2885623324,0sleep10-21swapper/118:35:461
28405070,0irq/26-eth0-rx-31086-21perl23:24:173
4160,0ktimersoftd/00-21swapper/023:27:040
339960,0migration/35045-21sh23:04:383
339960,0migration/34322-21missed_timers19:29:243
339960,0migration/327265-21sh21:19:113
339960,0migration/326906-21sh23:59:383
339960,0migration/326768-21irqrtprio21:59:213
339960,0migration/31553-21sh21:43:523
339960,0migration/313831-21ntp_offset22:29:243
339960,0migration/312185-21runrttasks23:52:023
292099965,0cyclictest4290-21ssh23:47:020
292099960,0cyclictest0-21swapper/023:30:000
292099960,0cyclictest0-21swapper/023:16:360
292099960,0cyclictest0-21swapper/023:16:360
28415060,0irq/27-eth0-tx-2819-21ssh00:06:192
28415060,0irq/27-eth0-tx-2698-21ssh21:03:232
28415060,0irq/27-eth0-tx-26153-21ssh20:55:532
28415060,0irq/27-eth0-tx-15582-21if_eth123:14:202
28415060,0irq/27-eth0-tx-15582-21if_eth123:14:202
28415060,0irq/27-eth0-tx-0-21swapper/223:35:502
28415060,0irq/27-eth0-tx-0-21swapper/222:52:152
28415060,0irq/27-eth0-tx-0-21swapper/222:39:512
28415060,0irq/27-eth0-tx-0-21swapper/222:33:242
28415060,0irq/27-eth0-tx-0-21swapper/222:25:332
28415060,0irq/27-eth0-tx-0-21swapper/222:18:412
28415060,0irq/27-eth0-tx-0-21swapper/222:13:502
28415060,0irq/27-eth0-tx-0-21swapper/222:03:512
28415060,0irq/27-eth0-tx-0-21swapper/221:58:302
28415060,0irq/27-eth0-tx-0-21swapper/221:51:512
28415060,0irq/27-eth0-tx-0-21swapper/221:37:122
28415060,0irq/27-eth0-tx-0-21swapper/221:33:032
28415060,0irq/27-eth0-tx-0-21swapper/221:14:442
28415060,0irq/27-eth0-tx-0-21swapper/221:13:472
28415060,0irq/27-eth0-tx-0-21swapper/221:08:342
28415060,0irq/27-eth0-tx-0-21swapper/220:46:332
28415060,0irq/27-eth0-tx-0-21swapper/220:41:312
28415060,0irq/27-eth0-tx-0-21swapper/200:01:052
28405060,0irq/26-eth0-rx-732-21pmu-power20:59:273
28405060,0irq/26-eth0-rx-0-21swapper/321:09:073
27160,0ktimersoftd/227365-21diskmemload23:27:042
249960,0watchdog/20-21swapper/223:56:182
169960,0watchdog/10-21swapper/122:35:151
149960,0migration/032183-21latency_hist19:54:040
149960,0migration/024334-21sh22:18:370
149960,0migration/013323-21switchtime20:24:280
8225050,0irq/24-0000:00:0-21swapper/219:24:042
339950,0migration/38339-21munin-node21:24:223
339950,0migration/37861-21cpuspeed_turbos20:24:183
339950,0migration/36497-21kthreadcore19:09:233
339950,0migration/3476-21kthreadcore23:44:233
339950,0migration/328909-21kthreadcore22:39:233
339950,0migration/327897-21kthreadcore18:59:233
339950,0migration/325652-21runrttasks21:14:453
339950,0migration/325110-21munin-node22:19:183
339950,0migration/324050-21taskset19:19:483
339950,0migration/320610-21kthreadcore19:44:223
339950,0migration/320470-21taskset20:31:363
339950,0migration/317641-21fschecks_time18:54:193
339950,0migration/31659-21fschecks_time22:04:193
339950,0migration/316255-21munin-node21:29:263
339950,0migration/312620-21kthreadcore22:49:233
339950,0migration/312105-21latency23:09:223
339950,0migration/311866-21munin-node20:49:173
339950,0migration/311111-21sh19:34:253
292119954,0cyclictest3425-21lldpd20:38:442
292109950,0cyclictest0-21swapper/122:32:171
292109950,0cyclictest0-21swapper/121:53:401
292099955,0cyclictest0-21swapper/022:32:560
292099954,0cyclictest6445-21snmpd23:49:300
292099950,0cyclictest0-21swapper/023:09:120
292099950,0cyclictest0-21swapper/020:47:090
28415050,0irq/27-eth0-tx-6778-21munin-node23:09:282
28415050,0irq/27-eth0-tx-6627-21munin-node19:59:200
28415050,0irq/27-eth0-tx-32284-21munin-node19:54:240
28415050,0irq/27-eth0-tx-23313-21munin-node22:59:302
28415050,0irq/27-eth0-tx-13277-21munin-node20:04:260
28415050,0irq/27-eth0-tx-0-21swapper/223:40:172
28415050,0irq/27-eth0-tx-0-21swapper/223:20:192
28415050,0irq/27-eth0-tx-0-21swapper/223:05:412
28415050,0irq/27-eth0-tx-0-21swapper/222:47:352
28415050,0irq/27-eth0-tx-0-21swapper/222:06:272
28415050,0irq/27-eth0-tx-0-21swapper/221:41:082
28415050,0irq/27-eth0-tx-0-21swapper/221:24:262
28415050,0irq/27-eth0-tx-0-21swapper/221:21:002
28415050,0irq/27-eth0-tx-0-21swapper/220:50:592
28415050,0irq/27-eth0-tx-0-21swapper/020:23:020
28415050,0irq/27-eth0-tx-0-21swapper/018:40:180
28405050,0irq/26-eth0-rx-6315-21kthreadcore20:44:223
28405050,0irq/26-eth0-rx-4191-21kthreadcore20:19:233
28405050,0irq/26-eth0-rx-30905-21kthreadcore00:04:213
28405050,0irq/26-eth0-rx-3067-1auditd22:24:043
28405050,0irq/26-eth0-rx-2071-21ssh20:41:423
28405050,0irq/26-eth0-rx-20409-21cut21:54:243
28405050,0irq/26-eth0-rx-20136-21ssh23:58:073
28405050,0irq/26-eth0-rx-10069-21cpu21:49:183
28405050,0irq/26-eth0-rx-0-21swapper/323:38:283
28405050,0irq/26-eth0-rx-0-21swapper/323:30:313
28405050,0irq/26-eth0-rx-0-21swapper/323:22:313
28405050,0irq/26-eth0-rx-0-21swapper/322:46:163
28405050,0irq/26-eth0-rx-0-21swapper/322:38:583
28405050,0irq/26-eth0-rx-0-21swapper/322:17:033
28405050,0irq/26-eth0-rx-0-21swapper/322:10:433
28405050,0irq/26-eth0-rx-0-21swapper/321:04:173
259950,0migration/26932-21if_eth023:29:202
259950,0migration/25667-21fschecks_count23:49:192
259950,0migration/231562-21ksmtuned22:21:362
259950,0migration/230053-21munin-node23:44:182
259950,0migration/228665-21kthreadcore20:14:232
259950,0migration/218990-21kthreadcore18:54:222
259950,0migration/216413-21taskset18:49:282
259950,0migration/216217-21sh22:54:172
259950,0migration/212309-21kthreadcore20:24:232
249950,0watchdog/22608-21perl21:44:192
19150,0ktimersoftd/10-21swapper/121:59:111
179950,0migration/19197-21munin-node18:44:231
169950,0watchdog/16667-21grep21:44:231
169950,0watchdog/16185-21kthreadcore19:09:231
169950,0watchdog/131301-21if_err_eth1.323:24:191
169950,0watchdog/122623-21cut19:19:231
169950,0watchdog/10-21swapper/121:14:191
169950,0watchdog/10-21swapper/120:41:471
169950,0watchdog/10-21swapper/100:01:181
149950,0migration/09682-21kthreadcore19:34:220
149950,0migration/08861-21kthreadcore18:44:220
149950,0migration/07844-21kthreadcore21:04:210
149950,0migration/07502-21taskset22:49:090
149950,0migration/04510-21kthreadcore21:24:200
149950,0migration/030047-21kthreadcore21:59:230
149950,0migration/029229-21kthreadcore20:39:240
149950,0migration/026718-21kthreadcore23:39:290
149950,0migration/023775-21kthreadcore21:14:220
149950,0migration/019202-21fschecks_time21:34:190
149950,0migration/017252-21sh21:30:390
149950,0migration/016476-21kthreadcore20:49:220
149950,0migration/01463-21kthreadcore19:04:220
149950,0migration/014351-21aten_r1power_fr20:29:170
149950,0migration/011642-21cpuspeed_turbos21:09:180
13509250,0sleep113511-21ssh23:11:241
339940,0migration/330443-21timerandwakeup19:24:283
339940,0migration/323247-21kthreadcore20:09:253
339940,0migration/315422-21sh22:54:123
339940,0migration/315416-21kthreadcore19:39:263
339940,0migration/315399-21kthreadcore18:49:223
339940,0migration/314694-21sh23:14:163
339940,0migration/314694-21sh23:14:163
292119943,0cyclictest6445-21snmpd20:33:292
292119943,0cyclictest6445-21snmpd19:35:292
292119943,0cyclictest6445-21snmpd19:18:582
292119943,0cyclictest3425-21lldpd19:25:132
292119943,0cyclictest32183-21awk19:29:212
292119943,0cyclictest30275-21awk18:39:192
292119943,0cyclictest20805-21awk20:09:222
292119943,0cyclictest19918-21awk19:44:212
292119943,0cyclictest14210-21awk20:04:202
292119943,0cyclictest13165-21awk19:39:232
292109943,0cyclictest6445-21snmpd23:40:301
292109943,0cyclictest6445-21snmpd22:59:141
292109943,0cyclictest6445-21snmpd22:21:591
292109943,0cyclictest6445-21snmpd19:27:141
292109943,0cyclictest3691-21sadc00:09:031
292109943,0cyclictest28242-21awk21:19:211
292109943,0cyclictest26977-21kworker/1:223:50:181
292109943,0cyclictest26977-21kworker/1:223:44:061
292109943,0cyclictest26977-21kworker/1:223:35:181
292109943,0cyclictest26977-21kworker/1:223:22:061
292109943,0cyclictest26977-21kworker/1:223:15:021
292109943,0cyclictest26977-21kworker/1:223:15:021
292109943,0cyclictest26977-21kworker/1:222:55:001
292109943,0cyclictest26977-21kworker/1:222:51:361
292109943,0cyclictest26977-21kworker/1:222:25:201
292109943,0cyclictest26977-21kworker/1:221:35:241
292109943,0cyclictest26977-21kworker/1:221:29:581
292109943,0cyclictest26977-21kworker/1:221:26:541
292109943,0cyclictest17807-21awk22:14:191
292109943,0cyclictest17787-21kworker/1:021:03:461
292109943,0cyclictest17787-21kworker/1:020:46:241
292109943,0cyclictest17787-21kworker/1:019:39:241
292109943,0cyclictest12275-21awk21:09:211
292099943,0cyclictest6445-21snmpd21:23:140
292099943,0cyclictest6445-21snmpd00:05:450
292099943,0cyclictest3425-21lldpd23:04:470
292099943,0cyclictest3425-21lldpd18:49:420
292099943,0cyclictest3110-21irqbalance22:36:080
292099943,0cyclictest3110-21irqbalance22:04:180
292099943,0cyclictest3110-21irqbalance21:40:480
292099943,0cyclictest3110-21irqbalance20:59:090
292099943,0cyclictest3110-21irqbalance19:45:190
292099943,0cyclictest27455-21awk20:14:210
292099943,0cyclictest25749-21awk22:19:210
292099943,0cyclictest24871-21awk22:39:200
292099943,0cyclictest24200-21awk22:59:190
292099943,0cyclictest21879-21awk20:34:210
292099940,0cyclictest0-21swapper/023:58:230
292099940,0cyclictest0-21swapper/021:44:410
292099940,0cyclictest0-21swapper/019:14:190
28415040,0irq/27-eth0-tx-0-21swapper/222:34:222
28415040,0irq/27-eth0-tx-0-21swapper/020:09:200
28415040,0irq/27-eth0-tx-0-21swapper/019:09:210
28415040,0irq/27-eth0-tx-0-21swapper/018:54:170
28405040,0irq/26-eth0-rx-22255-21diskstats23:39:243
28405040,0irq/26-eth0-rx-0-21swapper/322:59:383
28405040,0irq/26-eth0-rx-0-21swapper/321:45:463
28405040,0irq/26-eth0-rx-0-21swapper/321:37:053
28405040,0irq/26-eth0-rx-0-21swapper/320:57:053
28405040,0irq/26-eth0-rx-0-21swapper/319:59:183
28405040,0irq/26-eth0-rx-0-21swapper/318:44:213
259940,0migration/27740-21kthreadcore18:44:212
259940,0migration/26487-21kthreadcore19:09:232
259940,0migration/231661-21irqrtprio19:04:212
259940,0migration/227213-21kthreadcore19:49:212
259940,0migration/21671-21kthreadcore19:54:212
249940,0watchdog/226170-21cut18:59:232
179940,0migration/15482-21kthreadcore22:04:241
179940,0migration/15353-21kthreadcore20:19:241
179940,0migration/13103-21kthreadcore19:04:231
179940,0migration/123942-21latency_hist22:39:041
179940,0migration/120888-21latency_hist20:34:041
179940,0migration/114132-21irqcore23:54:241
179940,0migration/111957-21missed_timers19:59:251
169940,0watchdog/126171-21tr18:59:231
169940,0watchdog/123394-21kthreadcore20:54:231
169940,0watchdog/117772-21perl18:54:191
169940,0watchdog/114912-21grep19:14:231
169940,0watchdog/110540-21kthreadcore22:09:231
169940,0watchdog/10-21swapper/123:29:231
169940,0watchdog/10-21swapper/123:04:271
169940,0watchdog/10-21swapper/122:44:231
169940,0watchdog/10-21swapper/121:54:311
169940,0watchdog/10-21swapper/121:04:191
169940,0watchdog/10-21swapper/120:49:311
169940,0watchdog/10-21swapper/119:30:351
169940,0watchdog/10-21swapper/118:49:271
169940,0watchdog/10-21swapper/118:39:271
149940,0migration/05426-21switchtime22:44:310
149940,0migration/03633-21kthreadcore22:24:220
149940,0migration/030028-21missed_timers19:24:240
149940,0migration/027921-21kthreadcore18:59:230
149940,0migration/024990-21kthreadcore23:59:290
149940,0migration/023672-21timerwakeupswit21:54:290
149940,0migration/022171-21runrttasks22:56:230
149940,0migration/02072-21kthreadcore19:29:230
149940,0migration/019788-21cpu20:54:180
149940,0migration/019145-21irqcore19:19:210
149940,0migration/016468-21taskset22:13:370
149940,0migration/016388-21kthreadcore19:39:260
292119932,0cyclictest6445-21snmpd20:01:142
292119932,0cyclictest2423-21grep20:19:232
292109933,0cyclictest17787-21kworker/1:020:29:221
292109933,0cyclictest17787-21kworker/1:020:14:181
292109933,0cyclictest17787-21kworker/1:019:54:181
292109932,0cyclictest8600-21awk20:24:211
292109932,0cyclictest3425-21lldpd20:05:141
292109932,0cyclictest3425-21lldpd19:34:141
292109932,0cyclictest19829-21awk19:44:201
292099932,0cyclictest3425-21lldpd23:20:510
292099931,0cyclictest13375-21kthreadcore21:49:230
28405030,0irq/26-eth0-rx-0-21swapper/320:34:163
28405030,0irq/26-eth0-rx-0-21swapper/320:14:183
28405030,0irq/26-eth0-rx-0-21swapper/319:54:183
28405030,0irq/26-eth0-rx-0-21swapper/319:04:203
169930,0watchdog/126424-21munin-node19:49:191
169930,0watchdog/10-21swapper/121:39:191
169930,0watchdog/10-21swapper/120:09:231
339920,0migration/326648-21iostat_ios19:49:203
292099921,0cyclictest13900-21sh23:34:060
28405020,0irq/26-eth0-rx-0-21swapper/320:04:133
28405020,0irq/26-eth0-rx-0-21swapper/319:14:103
28405020,0irq/26-eth0-rx-0-21swapper/318:40:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional