You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 13:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Jan 02, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450620,0irq/25-eth10-21swapper/307:04:353
2493823625,0sleep00-21swapper/007:06:360
2481223526,0sleep20-21swapper/207:05:052
2488123425,0sleep10-21swapper/107:05:561
29715070,0irq/26-eth0-rx-30488-21ssh09:10:443
4160,0ktimersoftd/00-21swapper/010:25:290
35160,0ktimersoftd/30-21swapper/312:30:213
339960,0migration/39424-21sh12:10:533
339960,0migration/3941-21sh09:48:403
339960,0migration/38009-21sh09:33:413
339960,0migration/332574-21sh09:30:363
339960,0migration/332238-21sh11:45:323
339960,0migration/328732-21sh10:44:423
29715060,0irq/26-eth0-rx-28212-21munin-node12:23:193
29715060,0irq/26-eth0-rx-16593-21cut12:33:213
251479960,0cyclictest0-21swapper/211:48:212
251479960,0cyclictest0-21swapper/211:34:192
251479960,0cyclictest0-21swapper/210:06:012
251439966,0cyclictest0-21swapper/108:03:241
259960,0migration/212776-21sh10:36:282
179960,0migration/118778-21idleruntime11:58:211
149960,0migration/017745-21latency_hist11:58:050
4150,0ktimersoftd/00-21swapper/012:11:220
4150,0ktimersoftd/00-21swapper/010:20:100
339950,0migration/39498-21latency_hist11:53:053
339950,0migration/39018-21kthreadcore07:43:253
339950,0migration/38550-21kthreadcore10:53:213
339950,0migration/3577-21munin-node07:13:233
339950,0migration/34566-21df_abs10:13:193
339950,0migration/332596-21taskset07:35:573
339950,0migration/32804-21unixbench_singl08:48:293
339950,0migration/324551-21munin-node12:18:243
339950,0migration/323367-21latency_hist09:08:053
339950,0migration/321200-21taskset08:15:433
339950,0migration/320109-21taskset07:49:013
339950,0migration/31856-21cpuspeed_turbos08:03:183
339950,0migration/317120-21kthreadcore10:58:213
339950,0migration/316418-21kthreadcore10:38:223
339950,0migration/315887-21munin-node08:58:283
339950,0migration/312613-21taskset07:21:293
339950,0migration/31133-21fschecks_count07:38:183
339950,0migration/31071-21kthreadcore10:48:233
29725050,0irq/27-eth0-tx-3078-21munin-node09:53:152
29725050,0irq/27-eth0-tx-29816-21open_files11:23:252
29725050,0irq/27-eth0-tx-27407-21taskset10:24:112
29725050,0irq/27-eth0-tx-25253-21kthreadcore10:43:222
29725050,0irq/27-eth0-tx-16869-21munin-node09:03:222
29725050,0irq/27-eth0-tx-1113-21ssh12:06:132
29725050,0irq/27-eth0-tx-10635-21munin-node12:13:242
29725050,0irq/27-eth0-tx-0-21swapper/210:56:242
29725050,0irq/27-eth0-tx-0-21swapper/210:48:052
29725050,0irq/27-eth0-tx-0-21swapper/209:49:582
29725050,0irq/27-eth0-tx-0-21swapper/209:13:222
29715050,0irq/26-eth0-rx-6254-21timerwakeupswit11:28:283
29715050,0irq/26-eth0-rx-26458-21ssh09:47:123
29715050,0irq/26-eth0-rx-26173-21latency_hist12:03:053
29715050,0irq/26-eth0-rx-23295-21grep10:23:203
29715050,0irq/26-eth0-rx-19189-21cut11:38:213
29715050,0irq/26-eth0-rx-19082-21irqcore11:58:233
29715050,0irq/26-eth0-rx-1734-21cut10:28:243
29715050,0irq/26-eth0-rx-13589-21if_eth1.310:18:203
29715050,0irq/26-eth0-rx-0-21swapper/311:48:223
29715050,0irq/26-eth0-rx-0-21swapper/311:33:463
29715050,0irq/26-eth0-rx-0-21swapper/311:18:233
29715050,0irq/26-eth0-rx-0-21swapper/311:07:543
29715050,0irq/26-eth0-rx-0-21swapper/310:33:223
29715050,0irq/26-eth0-rx-0-21swapper/310:09:473
29715050,0irq/26-eth0-rx-0-21swapper/310:04:173
29715050,0irq/26-eth0-rx-0-21swapper/309:58:143
29715050,0irq/26-eth0-rx-0-21swapper/309:38:223
29715050,0irq/26-eth0-rx-0-21swapper/309:23:243
29715050,0irq/26-eth0-rx-0-21swapper/309:19:513
27150,0ktimersoftd/20-21swapper/212:11:222
27150,0ktimersoftd/20-21swapper/209:28:302
251479955,0cyclictest0-21swapper/210:13:262
251479950,0cyclictest0-21swapper/212:32:012
251479950,0cyclictest0-21swapper/212:26:582
251479950,0cyclictest0-21swapper/211:58:542
251479950,0cyclictest0-21swapper/211:40:562
251439955,0cyclictest0-21swapper/111:23:181
251439955,0cyclictest0-21swapper/110:47:401
251439950,0cyclictest0-21swapper/112:08:141
251439950,0cyclictest0-21swapper/111:18:151
251429955,0cyclictest0-21swapper/010:54:510
251429955,0cyclictest0-21swapper/010:42:330
251429950,0cyclictest0-21swapper/011:45:560
259950,0migration/29079-21taskset09:35:582
259950,0migration/24813-21pmu-power11:08:262
259950,0migration/23179-21sh10:28:382
259950,0migration/229684-21sh11:04:202
259950,0migration/229217-21kthreadcore07:08:242
259950,0migration/229110-21latency_hist08:48:052
259950,0migration/222221-21sh11:18:582
259950,0migration/21989-21sh10:08:372
259950,0migration/219609-21latency_hist07:28:052
259950,0migration/219411-21fschecks_time09:43:192
259950,0migration/216106-21kthreadcore09:58:212
259950,0migration/215340-21sh09:20:452
259950,0migration/211466-21irqcore09:38:212
249950,0watchdog/27951-21diskstats11:13:192
19150,0ktimersoftd/10-21swapper/110:29:231
19150,0ktimersoftd/10-21swapper/110:25:291
19150,0ktimersoftd/10-21swapper/109:48:141
179950,0migration/15594-21kthreadcore07:38:231
179950,0migration/15477-21kthreadcore08:53:241
179950,0migration/122801-21kthreadcore10:03:221
179950,0migration/118443-21kthreadcore09:23:241
179950,0migration/117259-21munin-node12:33:231
179950,0migration/111191-21dump-pmu-power09:58:051
149950,0migration/09353-21latency_hist11:53:050
149950,0migration/08061-21taskset08:07:410
149950,0migration/06888-21taskset07:40:190
149950,0migration/06517-21irqcore10:33:220
149950,0migration/0546-21kthreadcore11:08:220
149950,0migration/03808-21kthreadcore09:13:230
149950,0migration/03438-21sh10:12:020
149950,0migration/029282-21kthreadcore12:03:210
149950,0migration/029058-21munin-node11:03:270
149950,0migration/028830-21kthreadcore07:33:220
149950,0migration/028348-21munin-node07:58:210
149950,0migration/02759-21sh10:28:280
149950,0migration/026337-21kthreadcore11:23:220
149950,0migration/024804-21sh12:00:130
149950,0migration/020933-21kthreadcore12:18:220
149950,0migration/019831-21kthreadcore11:38:210
149950,0migration/01899-21taskset09:51:010
149950,0migration/017855-21sh09:41:340
149950,0migration/01158-21sh09:32:440
149950,0migration/011244-21fschecks_time12:13:190
149950,0migration/010736-21kthreadcore08:08:210
339940,0migration/35825-21diskmemload09:13:473
339940,0migration/325205-21kthreadcore08:43:243
339940,0migration/319171-21taskset07:25:053
29725040,0irq/27-eth0-tx-6559-21runrttasks08:03:282
29725040,0irq/27-eth0-tx-32592-21munin-node11:28:172
29725040,0irq/27-eth0-tx-31558-21ssh11:43:522
29725040,0irq/27-eth0-tx-11689-21ssh12:33:122
29725040,0irq/27-eth0-tx-0-21swapper/210:58:462
29725040,0irq/27-eth0-tx-0-21swapper/210:38:062
29725040,0irq/27-eth0-tx-0-21swapper/209:23:392
29725040,0irq/27-eth0-tx-0-21swapper/209:08:112
29725040,0irq/27-eth0-tx-0-21swapper/207:53:162
29725040,0irq/27-eth0-tx-0-21swapper/207:18:302
29715040,0irq/26-eth0-rx-7785-21grep09:53:223
29715040,0irq/26-eth0-rx-0-21swapper/312:13:173
29715040,0irq/26-eth0-rx-0-21swapper/311:23:093
29715040,0irq/26-eth0-rx-0-21swapper/311:13:233
29715040,0irq/26-eth0-rx-0-21swapper/311:09:403
251479943,0cyclictest3514-21snmpd08:08:152
251479943,0cyclictest3514-21snmpd07:52:152
251479943,0cyclictest3514-21snmpd07:41:302
251439944,0cyclictest0-21swapper/112:16:261
251439944,0cyclictest0-21swapper/111:13:131
251439944,0cyclictest0-21swapper/110:09:411
251439943,0cyclictest3556-21lldpd12:04:111
251439943,0cyclictest3556-21lldpd11:28:121
251439943,0cyclictest3556-21lldpd10:55:111
251439943,0cyclictest3556-21lldpd10:20:411
251439943,0cyclictest3556-21lldpd10:13:111
251439943,0cyclictest3556-21lldpd09:46:411
251439943,0cyclictest23513-21awk08:43:221
251439943,0cyclictest17645-21sadc11:58:051
251439943,0cyclictest13952-21awk07:23:211
251429944,0cyclictest0-21swapper/011:20:140
251429944,0cyclictest0-21swapper/010:03:550
251429943,0cyclictest8404-21awk11:13:210
251429943,0cyclictest3556-21lldpd07:28:410
251429943,0cyclictest3514-21snmpd12:29:000
251429943,0cyclictest3514-21snmpd07:17:450
251429943,0cyclictest31439-21awk10:48:210
251429943,0cyclictest30081-21awk08:48:210
251429943,0cyclictest29165-21awk08:23:200
251429943,0cyclictest28253-21awk12:23:190
251429943,0cyclictest2476-21awk09:33:190
251429943,0cyclictest23465-21awk08:43:210
251429943,0cyclictest15866-21awk08:13:250
251429943,0cyclictest11113-21awk08:58:200
259940,0migration/29803-21runrttasks08:56:142
259940,0migration/29294-21kthreadcore07:43:252
259940,0migration/23543-21kthreadcore07:13:242
259940,0migration/227986-21runrttasks08:21:562
259940,0migration/221805-21kthreadcore12:18:222
259940,0migration/220029-21kthreadcore08:38:222
259940,0migration/216219-21munin-node08:58:302
259940,0migration/215797-21munin-node08:13:242
259940,0migration/215487-21timerandwakeup11:53:292
259940,0migration/211056-21kthreadcore08:33:202
249940,0watchdog/218683-21python07:23:282
249940,0watchdog/21574-21grep08:48:242
249940,0watchdog/212529-21ssh10:18:072
179940,0migration/17763-21df_abs07:43:211
179940,0migration/16086-21gltestperf10:33:191
179940,0migration/1500-21kthreadcore08:48:231
179940,0migration/14332-21kthreadcore11:08:241
179940,0migration/132070-21kthreadcore12:23:221
179940,0migration/132024-21sendmail_mailtr07:33:271
179940,0migration/129626-21sh11:04:111
179940,0migration/129465-21kthreadcore07:58:211
179940,0migration/129318-21sendmail_mailqu09:08:261
179940,0migration/127921-21kthreadcore11:43:241
179940,0migration/126066-21fschecks_time09:28:191
179940,0migration/123675-21kthreadcore12:18:231
179940,0migration/122556-21pmu-power11:38:251
179940,0migration/119405-21kthreadcore09:03:211
179940,0migration/118619-21kthreadcore10:38:231
179940,0migration/111602-21kthreadcore08:58:221
169940,0watchdog/19689-21df_inode08:33:161
169940,0watchdog/129175-21if_err_eth108:23:201
169940,0watchdog/128015-21kthreadcore07:08:241
169940,0watchdog/121115-21ssh10:58:431
169940,0watchdog/120844-21irqrtprio07:28:241
169940,0watchdog/119609-21kthreadcore08:13:281
169940,0watchdog/11648-21fschecks_count11:48:191
169940,0watchdog/11581-21cut10:48:231
169940,0watchdog/114485-21ssh09:18:511
169940,0watchdog/110373-21kthreadcore11:33:231
169940,0watchdog/10-21swapper/109:33:311
169940,0watchdog/10-21swapper/107:48:241
169940,0watchdog/10-21swapper/107:13:201
149940,0migration/09512-21kthreadcore07:18:230
149940,0migration/07534-21kthreadcore09:53:220
149940,0migration/06108-21kthreadcore08:53:240
149940,0migration/029934-21sh09:09:170
149940,0migration/029294-21ksmtuned10:46:020
149940,0migration/025136-21kthreadcore08:18:230
149940,0migration/020292-21kthreadcore09:43:230
149940,0migration/017508-21fschecks_time09:03:170
149940,0migration/016316-21runrttasks09:22:470
29725030,0irq/27-eth0-tx-27486-21aten2.4-expect07:58:162
29715030,0irq/26-eth0-rx-0-21swapper/308:56:103
29715030,0irq/26-eth0-rx-0-21swapper/308:36:553
29715030,0irq/26-eth0-rx-0-21swapper/308:32:513
29715030,0irq/26-eth0-rx-0-21swapper/308:27:203
29715030,0irq/26-eth0-rx-0-21swapper/308:18:293
29715030,0irq/26-eth0-rx-0-21swapper/308:08:203
29715030,0irq/26-eth0-rx-0-21swapper/307:58:113
29715030,0irq/26-eth0-rx-0-21swapper/307:57:383
29715030,0irq/26-eth0-rx-0-21swapper/307:28:193
29715030,0irq/26-eth0-rx-0-21swapper/307:08:183
251479932,0cyclictest3531-21awk08:28:202
251479932,0cyclictest27260-21awk07:33:202
251439932,0cyclictest7205-21awk07:18:201
251439932,0cyclictest3514-21snmpd07:56:301
251439930,0cyclictest3717-21df_inode12:28:171
251439930,0cyclictest11455-21munin-node09:58:151
251439930,0cyclictest0-21swapper/109:13:161
251439930,0cyclictest0-21swapper/108:39:581
251429933,0cyclictest22656-21kworker/0:112:33:080
251429933,0cyclictest22656-21kworker/0:111:33:080
251429933,0cyclictest22656-21kworker/0:111:28:120
251429933,0cyclictest22656-21kworker/0:110:58:200
251429933,0cyclictest22656-21kworker/0:110:13:100
251429933,0cyclictest22656-21kworker/0:109:58:160
251429933,0cyclictest22656-21kworker/0:109:23:350
251429933,0cyclictest22656-21kworker/0:108:38:250
251429933,0cyclictest22656-21kworker/0:108:33:070
251429933,0cyclictest22656-21kworker/0:108:28:070
251429933,0cyclictest22656-21kworker/0:107:53:190
251429933,0cyclictest22656-21kworker/0:107:48:070
251429933,0cyclictest22656-21kworker/0:107:43:070
251429933,0cyclictest22656-21kworker/0:107:23:190
251429932,0cyclictest26033-21perl07:08:200
249930,0watchdog/20-21swapper/208:23:202
169930,0watchdog/10-21swapper/109:38:231
169930,0watchdog/10-21swapper/108:18:241
29715020,0irq/26-eth0-rx-0-21swapper/308:38:153
251439921,0cyclictest8321-21latency_hist08:08:051
169920,0watchdog/10-21swapper/108:28:081
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional