You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-07 - 22:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Sun Dec 07, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2448027069,0sleep10-21swapper/106:38:261
2429223826,0sleep30-21swapper/306:36:163
2442423422,0sleep00-21swapper/006:37:450
2432023329,0sleep20-21swapper/206:36:352
339960,0migration/37213-21missed_timers11:54:233
339960,0migration/35467-21sh09:51:443
339960,0migration/34460-21sh10:11:543
339960,0migration/34391-21sh10:34:103
339960,0migration/329960-21sh09:46:433
339960,0migration/322860-21sh09:21:203
339960,0migration/320300-21sh10:22:263
339960,0migration/317750-21sh11:00:113
339960,0migration/31232-21sh11:53:413
28415060,0irq/27-eth0-tx-30233-21ssh09:47:322
28415060,0irq/27-eth0-tx-0-21swapper/212:08:222
28415060,0irq/27-eth0-tx-0-21swapper/212:01:122
28415060,0irq/27-eth0-tx-0-21swapper/211:51:062
28415060,0irq/27-eth0-tx-0-21swapper/211:38:022
28415060,0irq/27-eth0-tx-0-21swapper/211:33:542
28415060,0irq/27-eth0-tx-0-21swapper/211:19:282
28415060,0irq/27-eth0-tx-0-21swapper/210:54:192
28415060,0irq/27-eth0-tx-0-21swapper/210:41:362
28415060,0irq/27-eth0-tx-0-21swapper/210:09:122
28415060,0irq/27-eth0-tx-0-21swapper/209:55:142
28415060,0irq/27-eth0-tx-0-21swapper/209:43:302
28415060,0irq/27-eth0-tx-0-21swapper/209:25:202
28415060,0irq/27-eth0-tx-0-21swapper/209:20:422
28415060,0irq/27-eth0-tx-0-21swapper/209:16:402
28415060,0irq/27-eth0-tx-0-21swapper/209:13:172
28415060,0irq/27-eth0-tx-0-21swapper/208:44:322
28415060,0irq/27-eth0-tx-0-21swapper/208:41:372
28415060,0irq/27-eth0-tx-0-21swapper/006:57:460
28405060,0irq/26-eth0-rx-9340-21cut09:34:233
28405060,0irq/26-eth0-rx-20700-21sh10:00:103
27160,0ktimersoftd/20-21swapper/209:35:332
246069960,0cyclictest0-21swapper/011:11:040
19160,0ktimersoftd/10-21swapper/112:01:161
169960,0watchdog/121698-21yum08:09:301
169960,0watchdog/10-21swapper/111:23:331
169960,0watchdog/10-21swapper/110:26:461
149960,0migration/02677-21if_eth1.311:54:190
8225050,0irq/24-0000:00:3039-21munin-run08:24:032
4150,0ktimersoftd/00-21swapper/011:40:050
339950,0migration/38006-21kthreadcore10:14:213
339950,0migration/37510-21kthreadcore06:49:223
339950,0migration/34501-21missed_timers06:44:243
339950,0migration/33367-21kthreadcore09:29:223
339950,0migration/332667-21aten_r1power_fr07:09:143
339950,0migration/330594-21kthreadcore07:54:233
339950,0migration/327998-21munin-node11:09:183
339950,0migration/323017-21kthreadcore10:44:223
339950,0migration/322586-21kernelversion07:49:243
339950,0migration/320096-21kthreadcore08:09:233
339950,0migration/319694-21fschecks_count06:59:243
339950,0migration/316071-21missed_timers11:19:253
339950,0migration/315945-21unixbench_multi11:59:293
339950,0migration/314891-21unixbench-2d08:04:323
339950,0migration/313696-21munin-node07:39:273
339950,0migration/313346-21kthreadcore08:29:193
339950,0migration/310399-21kthreadcore09:14:223
28415050,0irq/27-eth0-tx-2933-40yum-updatesd-he08:23:020
28415050,0irq/27-eth0-tx-28821-21ssh10:29:062
28415050,0irq/27-eth0-tx-27961-21munin-node07:54:230
28415050,0irq/27-eth0-tx-26984-21munin-node07:29:230
28415050,0irq/27-eth0-tx-1658-21munin-node11:54:242
28415050,0irq/27-eth0-tx-13241-21munin-node10:19:282
28415050,0irq/27-eth0-tx-11217-21ssh11:39:182
28415050,0irq/27-eth0-tx-0-21swapper/211:15:322
28415050,0irq/27-eth0-tx-0-21swapper/211:10:322
28415050,0irq/27-eth0-tx-0-21swapper/211:07:582
28415050,0irq/27-eth0-tx-0-21swapper/210:45:242
28415050,0irq/27-eth0-tx-0-21swapper/210:34:222
28415050,0irq/27-eth0-tx-0-21swapper/210:26:312
28415050,0irq/27-eth0-tx-0-21swapper/210:02:462
28415050,0irq/27-eth0-tx-0-21swapper/209:53:232
28415050,0irq/27-eth0-tx-0-21swapper/209:06:262
28415050,0irq/27-eth0-tx-0-21swapper/208:58:332
28415050,0irq/27-eth0-tx-0-21swapper/208:50:462
28415050,0irq/27-eth0-tx-0-21swapper/008:39:260
28415050,0irq/27-eth0-tx-0-21swapper/006:59:220
28415050,0irq/27-eth0-tx-0-21swapper/006:52:450
28405050,0irq/26-eth0-rx-25798-21munin-run11:49:023
28405050,0irq/26-eth0-rx-24773-21ssh08:43:273
28405050,0irq/26-eth0-rx-12663-21grep11:39:193
28405050,0irq/26-eth0-rx-11830-21kthreadcore09:54:293
28405050,0irq/26-eth0-rx-10620-21kthreadcore08:54:223
28405050,0irq/26-eth0-rx-0-21swapper/311:29:063
28405050,0irq/26-eth0-rx-0-21swapper/311:24:493
28405050,0irq/26-eth0-rx-0-21swapper/310:54:223
28405050,0irq/26-eth0-rx-0-21swapper/310:43:453
28405050,0irq/26-eth0-rx-0-21swapper/310:06:553
28405050,0irq/26-eth0-rx-0-21swapper/309:04:103
28405050,0irq/26-eth0-rx-0-21swapper/308:24:203
28405050,0irq/26-eth0-rx-0-21swapper/307:04:213
259950,0migration/27286-21kthreadcore06:49:222
259950,0migration/224462-21sh11:45:072
259950,0migration/221842-21kthreadcore07:24:212
259950,0migration/218205-21kthreadcore08:59:222
259950,0migration/21320-21munin-node07:54:292
259950,0migration/213117-21fschecks_time06:54:172
246089955,0cyclictest0-21swapper/207:44:242
246089950,0cyclictest0-21swapper/210:51:042
246079955,0cyclictest0-21swapper/106:54:181
246079954,0cyclictest6445-21snmpd08:39:321
246079950,0cyclictest0-21swapper/111:03:051
246079950,0cyclictest0-21swapper/109:05:571
246069955,0cyclictest0-21swapper/008:49:110
246069955,0cyclictest0-21swapper/007:14:160
246069950,0cyclictest0-21swapper/009:54:550
246069950,0cyclictest0-21swapper/009:07:090
249950,0watchdog/2984-21kthreadcore06:44:222
249950,0watchdog/29142-21cut10:14:222
249950,0watchdog/232392-21fschecks_time09:29:182
249950,0watchdog/213591-21cut10:59:212
249950,0watchdog/20-21swapper/208:06:382
19150,0ktimersoftd/10-21swapper/109:31:381
179950,0migration/18680-21taskset08:03:091
179950,0migration/114731-21timerandwakeup08:54:281
169950,0watchdog/17863-21cat06:49:221
169950,0watchdog/15044-21kthreadcore10:54:211
169950,0watchdog/14574-21ssh10:12:181
169950,0watchdog/131437-21ssh09:28:421
169950,0watchdog/121058-21ssh10:01:181
169950,0watchdog/117156-21diskmemload09:11:261
169950,0watchdog/113194-21ssh10:39:211
169950,0watchdog/112391-21cut07:39:221
169950,0watchdog/10-21swapper/109:53:381
149950,0migration/08019-21kthreadcore08:24:200
149950,0migration/07206-21taskset09:10:580
149950,0migration/07006-21kthreadcore10:34:210
149950,0migration/06755-21kthreadcore10:54:220
149950,0migration/06662-21kthreadcore07:59:200
149950,0migration/05719-21sh09:52:330
149950,0migration/04836-21taskset10:13:080
149950,0migration/04801-21kthreadcore11:34:210
149950,0migration/031730-21timerandwakeup07:04:270
149950,0migration/029711-21kthreadcore10:49:200
149950,0migration/028909-21kthreadcore11:49:200
149950,0migration/02780-21kthreadcore09:29:210
149950,0migration/026894-21kthreadcore08:44:200
149950,0migration/024976-21kthreadcore07:24:220
149950,0migration/022532-21kthreadcore10:24:220
149950,0migration/022365-21sh09:19:440
149950,0migration/021445-21sh10:02:410
149950,0migration/020593-21kthreadcore11:44:210
149950,0migration/016939-21kthreadcore10:19:290
149950,0migration/015227-21kthreadcore10:59:220
149950,0migration/014665-21kthreadcore11:59:240
149950,0migration/014339-21unixbench-2d09:14:290
149950,0migration/012459-21kthreadcore08:04:210
149950,0migration/011788-21kthreadcore08:29:190
339940,0migration/35944-21kthreadcore08:49:233
339940,0migration/33408-21irqcore07:59:193
339940,0migration/332253-21kthreadcore08:19:223
339940,0migration/330269-21sh09:24:563
339940,0migration/329839-21kthreadcore07:29:203
339940,0migration/32956-21kthreadcore07:34:203
339940,0migration/328332-21kthreadcore08:44:203
339940,0migration/328007-21diskmemload10:49:033
339940,0migration/326475-21taskset07:25:563
339940,0migration/325660-21sh11:05:173
339940,0migration/323570-21diskmemload09:01:043
339940,0migration/320939-21taskset07:47:493
339940,0migration/313042-21fschecks_count06:54:173
28415040,0irq/27-eth0-tx-21297-21munin-node07:49:220
28415040,0irq/27-eth0-tx-0-21swapper/210:04:512
28415040,0irq/27-eth0-tx-0-21swapper/008:14:230
28415040,0irq/27-eth0-tx-0-21swapper/006:39:190
28405040,0irq/26-eth0-rx-0-21swapper/312:04:163
28405040,0irq/26-eth0-rx-0-21swapper/311:35:463
28405040,0irq/26-eth0-rx-0-21swapper/311:15:273
28405040,0irq/26-eth0-rx-0-21swapper/310:31:333
28405040,0irq/26-eth0-rx-0-21swapper/310:24:233
28405040,0irq/26-eth0-rx-0-21swapper/309:40:553
28405040,0irq/26-eth0-rx-0-21swapper/309:10:573
28405040,0irq/26-eth0-rx-0-21swapper/306:39:203
259940,0migration/22359-21kthreadcore07:09:202
259940,0migration/220061-21munin-node06:59:252
259940,0migration/216053-21runrttasks08:29:362
246089943,0cyclictest6445-21snmpd11:24:182
246089943,0cyclictest6445-21snmpd08:13:022
246089943,0cyclictest6445-21snmpd07:59:172
246089943,0cyclictest6445-21snmpd07:09:012
246089943,0cyclictest27826-21awk07:29:182
246089943,0cyclictest14514-21awk07:19:232
246089940,0cyclictest0-21swapper/207:54:032
246079944,0cyclictest0-21swapper/109:54:121
246079943,0cyclictest6445-21snmpd11:49:481
246079943,0cyclictest6445-21snmpd11:25:031
246079943,0cyclictest6445-21snmpd09:00:321
246079943,0cyclictest6445-21snmpd07:55:321
246079943,0cyclictest6445-21snmpd07:50:321
246079943,0cyclictest6445-21snmpd07:48:321
246079943,0cyclictest6445-21snmpd07:29:021
246079943,0cyclictest6445-21snmpd07:11:311
246079943,0cyclictest6445-21snmpd07:08:311
246079943,0cyclictest3425-21lldpd07:19:541
246079943,0cyclictest3110-21irqbalance10:18:271
246079943,0cyclictest29045-21awk10:49:191
246079943,0cyclictest24790-21kworker/1:211:54:311
246079943,0cyclictest23967-21awk09:44:171
246079943,0cyclictest20122-21awk11:04:181
246079943,0cyclictest1786-21awk08:49:211
246079943,0cyclictest11145-21awk11:39:171
246079940,0cyclictest0-21swapper/110:05:141
246069943,0cyclictest6445-21snmpd09:42:470
246069943,0cyclictest6392-21awk10:14:180
246069943,0cyclictest3593-21awk11:14:180
246069943,0cyclictest3425-21lldpd12:06:170
246069943,0cyclictest3110-21irqbalance08:55:570
246069943,0cyclictest29749-21awk10:29:190
246069943,0cyclictest26244-21sadc11:29:020
246069943,0cyclictest23002-21awk10:04:190
246069943,0cyclictest2121-21awk07:34:180
246069943,0cyclictest17747-21awk08:59:210
249940,0watchdog/20-21swapper/207:14:262
249940,0watchdog/20-21swapper/206:39:142
179940,0migration/19947-21kthreadcore09:14:221
179940,0migration/19198-21kthreadcore07:14:201
179940,0migration/18080-21fschecks_time09:34:201
179940,0migration/16057-21kthreadcore07:34:211
179940,0migration/14688-21kthreadcore11:34:211
179940,0migration/14664-21pmu-power06:44:251
179940,0migration/129003-21aten_r1power_cu08:19:151
179940,0migration/122796-21fschecks_count08:14:161
179940,0migration/121537-21runrttasks09:39:541
179940,0migration/118795-21kthreadcore09:19:211
179940,0migration/115232-21taskset08:06:571
169940,0watchdog/14853-21grep10:34:171
169940,0watchdog/131905-21ssh10:29:211
169940,0watchdog/123312-21sh12:04:211
169940,0watchdog/11045-21ssh11:10:051
169940,0watchdog/10-21swapper/111:44:171
169940,0watchdog/10-21swapper/111:29:171
169940,0watchdog/10-21swapper/111:14:331
169940,0watchdog/10-21swapper/110:47:411
169940,0watchdog/10-21swapper/110:19:061
169940,0watchdog/10-21swapper/108:44:141
169940,0watchdog/10-21swapper/108:38:421
169940,0watchdog/10-21swapper/108:24:261
169940,0watchdog/10-21swapper/106:59:301
149940,0migration/024140-21aten_r1power_vo09:24:170
149940,0migration/022565-21kthreadcore10:44:220
149940,0migration/015967-21kthreadcore07:19:250
149940,0migration/015783-21aten_r1power_po08:09:170
28405030,0irq/26-eth0-rx-0-21swapper/308:16:373
28405030,0irq/26-eth0-rx-0-21swapper/307:22:183
28405030,0irq/26-eth0-rx-0-21swapper/307:15:503
259930,0migration/228077-21timerandwakeup08:14:262
246089932,0cyclictest17154-21sadc08:39:032
246089930,0cyclictest1645-21diskstats07:34:162
246089930,0cyclictest10465-21cut07:39:212
246079932,0cyclictest10847-21awk08:29:171
246069932,0cyclictest8348-21awk09:34:210
246069932,0cyclictest30037-21sh09:46:570
246069932,0cyclictest11572-21awk11:19:210
246069930,0cyclictest6715-21munin-run07:14:030
246069930,0cyclictest291350irq/25-eth111:06:020
246069930,0cyclictest17156-21diskmemload11:29:240
246069930,0cyclictest0-21swapper/007:39:160
246069930,0cyclictest0-21swapper/006:44:260
249930,0watchdog/20-21swapper/208:24:262
169930,0watchdog/10-21swapper/107:29:221
169930,0watchdog/10-21swapper/106:39:101
149930,0migration/018972-21sh10:41:130
28405020,0irq/26-eth0-rx-0-21swapper/308:34:163
246069920,0cyclictest6372-21iceccd08:34:350
246069920,0cyclictest0-21swapper/007:44:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional