You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-11 - 15:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Tue Aug 11, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2924824544,0sleep00-21swapper/006:44:360
2936124028,0sleep30-21swapper/306:45:553
2323323424,0sleep10-21swapper/106:42:211
2907923127,0sleep20-21swapper/206:42:382
339960,0migration/36403-21sh11:35:283
339960,0migration/35636-21sh12:16:483
339960,0migration/330350-21unixbench-2d11:07:323
339960,0migration/324742-21sh10:03:343
339960,0migration/318727-21sh09:21:023
29725060,0irq/27-eth0-tx-29754-21munin-node06:47:260
29725060,0irq/27-eth0-tx-21353-21munin-node08:42:300
29715060,0irq/26-eth0-rx-3607-21aten2.4-expect09:12:253
29715060,0irq/26-eth0-rx-29841-21ssh12:11:093
29715060,0irq/26-eth0-rx-10104-21iostat_ios10:37:263
295389960,0cyclictest0-21swapper/211:10:362
295389960,0cyclictest0-21swapper/209:45:002
295389960,0cyclictest0-21swapper/207:32:342
259960,0migration/22102-21sh09:09:042
149960,0migration/023439-21sh11:05:190
149960,0migration/021925-21sh12:06:220
149960,0migration/018336-21sh09:19:490
149960,0migration/018005-21sh08:57:540
149960,0migration/014177-21sh11:39:480
149960,0migration/013954-21sh12:01:090
339950,0migration/38305-21latency09:32:293
339950,0migration/35329-21kthreadcore10:52:283
339950,0migration/35302-21kthreadcore08:52:273
339950,0migration/35205-21kthreadcore08:27:273
339950,0migration/332520-21latency_hist08:02:113
339950,0migration/332008-21kthreadcore09:07:273
339950,0migration/331659-21needreboot10:07:293
339950,0migration/327299-21kthreadcore11:47:283
339950,0migration/326983-21kthreadcore10:27:273
339950,0migration/323466-21taskset07:03:123
339950,0migration/322200-21taskset11:23:213
339950,0migration/320741-21kthreadcore09:42:243
339950,0migration/320703-21munin-node08:37:363
339950,0migration/319280-21latency_hist09:22:103
339950,0migration/318585-21munin-node07:47:333
339950,0migration/317941-21kthreadcore11:42:313
339950,0migration/316010-21latency06:57:283
339950,0migration/314129-21taskset08:33:313
339950,0migration/311279-21munin-node09:37:243
339950,0migration/310064-21kthreadcore07:17:263
29725050,0irq/27-eth0-tx-9081-21munin-node10:37:252
29725050,0irq/27-eth0-tx-6590-21ssh11:14:062
29725050,0irq/27-eth0-tx-6559-21runrttasks10:54:032
29725050,0irq/27-eth0-tx-4084-21munin-node06:52:270
29725050,0irq/27-eth0-tx-31717-21munin-node07:37:290
29725050,0irq/27-eth0-tx-1778-21ssh08:47:480
29725050,0irq/27-eth0-tx-16979-21ssh09:59:102
29725050,0irq/27-eth0-tx-0-21swapper/008:22:290
29725050,0irq/27-eth0-tx-0-21swapper/008:22:290
29725050,0irq/27-eth0-tx-0-21swapper/006:59:060
29715050,0irq/26-eth0-rx-6921-21sensors10:32:323
29715050,0irq/26-eth0-rx-20610-21kthreadcore11:02:293
29715050,0irq/26-eth0-rx-20458-21kthreadcore07:27:283
29715050,0irq/26-eth0-rx-2019-21ssh09:52:053
29715050,0irq/26-eth0-rx-17034-21cut10:42:213
29715050,0irq/26-eth0-rx-15901-21munin-node09:57:293
29715050,0irq/26-eth0-rx-0-21swapper/311:52:263
29715050,0irq/26-eth0-rx-0-21swapper/311:41:433
29715050,0irq/26-eth0-rx-0-21swapper/311:20:073
29715050,0irq/26-eth0-rx-0-21swapper/310:12:133
29715050,0irq/26-eth0-rx-0-21swapper/309:55:433
29715050,0irq/26-eth0-rx-0-21swapper/309:05:423
29715050,0irq/26-eth0-rx-0-21swapper/308:22:233
29715050,0irq/26-eth0-rx-0-21swapper/308:22:233
295389955,0cyclictest15862-21kthreadcore08:12:272
295389955,0cyclictest0-21swapper/211:57:162
295389955,0cyclictest0-21swapper/210:07:232
295389955,0cyclictest0-21swapper/209:50:032
295389955,0cyclictest0-21swapper/209:39:572
295389955,0cyclictest0-21swapper/209:33:482
295389955,0cyclictest0-21swapper/209:21:212
295389955,0cyclictest0-21swapper/209:04:102
295389954,0cyclictest3514-21snmpd07:13:122
295389950,0cyclictest0-21swapper/210:28:502
295389950,0cyclictest0-21swapper/209:12:122
295389950,0cyclictest0-21swapper/206:52:122
295379955,0cyclictest0-21swapper/111:20:151
295379954,0cyclictest540-21tr07:37:281
295379950,0cyclictest0-21swapper/111:59:121
295379950,0cyclictest0-21swapper/109:55:541
259950,0migration/24964-21sh12:14:452
259950,0migration/230255-21sh11:50:372
259950,0migration/223313-21idleruntime12:07:252
259950,0migration/221118-21gltestperf08:17:232
259950,0migration/221035-21kthreadcore07:27:282
249950,0watchdog/26098-21kthreadcore09:52:282
249950,0watchdog/223615-21ssh10:44:002
179950,0migration/18803-21timerandwakeup09:32:341
179950,0migration/17841-21latency_hist08:32:111
179950,0migration/16992-21sh11:15:171
179950,0migration/13778-21kthreadcore10:32:281
179950,0migration/127328-21munin-node08:42:341
179950,0migration/12646-21if_eth010:12:261
179950,0migration/122970-21latency_hist11:47:091
179950,0migration/11999-21kthreadcore06:47:281
179950,0migration/116054-21kthreadcore08:12:271
179950,0migration/111339-21taskset08:56:511
179950,0migration/111180-21taskset07:18:401
179950,0migration/110863-21sh09:37:211
169950,0watchdog/121758-21tr07:02:291
169950,0watchdog/116809-21df_inode11:02:241
169950,0watchdog/10-21swapper/111:37:281
169950,0watchdog/10-21swapper/108:37:281
149950,0migration/0936-21fschecks_count10:52:230
149950,0migration/07837-21taskset11:17:100
149950,0migration/07756-21latency09:52:290
149950,0migration/05565-21kthreadcore10:12:280
149950,0migration/03711-21munin-node09:12:250
149950,0migration/032340-21runrttasks11:52:280
149950,0migration/031981-21taskset10:28:240
149950,0migration/031313-21latency07:57:280
149950,0migration/030773-21dump-pmu-power11:52:090
149950,0migration/03037-21kthreadcore08:27:270
149950,0migration/029898-21timerwakeupswit07:07:350
149950,0migration/026930-21latency_hist08:22:110
149950,0migration/024201-21timerandwakeup10:02:340
149950,0migration/023825-21sh10:22:440
149950,0migration/020600-21mdstat11:42:320
149950,0migration/018397-21aten_r1power_cu07:27:220
149950,0migration/017437-21aten_r1power_en07:02:220
149950,0migration/017177-21munin-node10:42:220
149950,0migration/01601-21fschecks_count10:32:240
149950,0migration/015886-21kthreadcore07:22:290
149950,0migration/012605-21pmu-power08:07:300
149950,0migration/012419-21kthreadcore10:57:270
149950,0migration/011186-21kthreadcore10:37:270
8215040,0irq/24-0000:00:4982-21cat07:17:112
8215040,0irq/24-0000:00:12635-21cat07:47:112
339940,0migration/38798-21kthreadcore06:52:283
339940,0migration/38379-21latency_hist10:57:093
339940,0migration/330377-21taskset07:10:503
339940,0migration/324962-21aten2.4-expect10:47:213
339940,0migration/321287-21kthreadcore10:22:283
339940,0migration/32030-21kthreadcore06:47:283
339940,0migration/318153-21kthreadcore12:02:273
339940,0migration/317779-21kthreadcore08:12:283
29725040,0irq/27-eth0-tx-4848-21ssh11:52:422
29725040,0irq/27-eth0-tx-4553-21ssh08:52:252
29725040,0irq/27-eth0-tx-0-21swapper/212:02:202
29725040,0irq/27-eth0-tx-0-21swapper/211:27:162
29725040,0irq/27-eth0-tx-0-21swapper/211:17:332
29725040,0irq/27-eth0-tx-0-21swapper/211:02:132
29725040,0irq/27-eth0-tx-0-21swapper/210:33:022
29725040,0irq/27-eth0-tx-0-21swapper/210:22:142
29725040,0irq/27-eth0-tx-0-21swapper/210:12:252
29725040,0irq/27-eth0-tx-0-21swapper/008:32:240
29725040,0irq/27-eth0-tx-0-21swapper/007:44:140
29725040,0irq/27-eth0-tx-0-21swapper/007:12:250
29715040,0irq/26-eth0-rx-31433-21taskset07:36:423
29715040,0irq/26-eth0-rx-0-21swapper/311:57:133
29715040,0irq/26-eth0-rx-0-21swapper/311:27:133
29715040,0irq/26-eth0-rx-0-21swapper/311:12:423
29715040,0irq/26-eth0-rx-0-21swapper/310:19:013
29715040,0irq/26-eth0-rx-0-21swapper/309:29:353
29715040,0irq/26-eth0-rx-0-21swapper/308:57:163
29715040,0irq/26-eth0-rx-0-21swapper/308:48:043
29715040,0irq/26-eth0-rx-0-21swapper/308:02:243
29715040,0irq/26-eth0-rx-0-21swapper/307:41:383
295389943,0cyclictest3556-21lldpd10:47:282
295389943,0cyclictest3556-21lldpd08:38:282
295389943,0cyclictest3556-21lldpd08:08:282
295389943,0cyclictest26963-21awk07:57:252
295389943,0cyclictest20294-21awk09:22:262
295389943,0cyclictest20255-21awk07:52:252
295389943,0cyclictest16813-21awk11:22:262
295389943,0cyclictest16181-21awk11:42:292
295379943,0cyclictest9424-21awk10:57:241
295379943,0cyclictest3514-21snmpd11:52:131
295379943,0cyclictest3514-21snmpd10:49:431
295379943,0cyclictest26987-21awk07:57:251
295379943,0cyclictest24989-21awk07:07:271
295379943,0cyclictest24617-21awk11:27:221
295379943,0cyclictest15477-21awk12:02:251
295379943,0cyclictest1353-21awk10:52:251
295369943,0cyclictest724-21awk11:12:250
295369943,0cyclictest3556-21lldpd09:48:590
295369943,0cyclictest3514-21snmpd09:31:580
295369943,0cyclictest20178-21kworker/0:211:32:210
295369943,0cyclictest20178-21kworker/0:210:50:290
295369943,0cyclictest20178-21kworker/0:210:18:090
295369943,0cyclictest20178-21kworker/0:210:09:550
295369943,0cyclictest20178-21kworker/0:209:40:510
295369943,0cyclictest20178-21kworker/0:209:08:090
295369943,0cyclictest14591-21awk08:12:250
295369943,0cyclictest11294-21awk09:57:240
259940,0migration/29727-21sh10:17:162
259940,0migration/231633-21runrttasks11:32:142
259940,0migration/229843-21kthreadcore09:27:282
259940,0migration/22615-21latency06:47:282
259940,0migration/219752-21kthreadcore07:02:282
259940,0migration/218405-21aten_r1power_vo10:02:222
259940,0migration/214800-21kthreadcore08:57:272
259940,0migration/213013-21kthreadcore06:57:272
249940,0watchdog/28976-21kthreadcore11:37:282
249940,0watchdog/28534-21sh10:57:202
249940,0watchdog/232403-21fschecks_time07:37:252
249940,0watchdog/232033-21kthreadcore08:47:282
249940,0watchdog/227030-21munin-node08:22:202
249940,0watchdog/227030-21munin-node08:22:202
249940,0watchdog/22083-21idleruntime08:27:242
249940,0watchdog/20-21swapper/207:22:252
179940,0migration/18448-21munin-node08:32:241
179940,0migration/16293-21runrttasks11:35:101
179940,0migration/14542-21kthreadcore08:02:311
179940,0migration/14230-21gltestperf09:12:271
179940,0migration/129745-21kthreadcore08:22:261
179940,0migration/129745-21kthreadcore08:22:261
179940,0migration/129484-21kthreadcore11:07:271
179940,0migration/127771-21fschecks_count09:27:241
179940,0migration/126994-21kthreadcore10:27:271
179940,0migration/117338-21taskset10:00:141
179940,0migration/116972-21taskset07:00:031
179940,0migration/116165-21taskset10:40:231
179940,0migration/113908-21kthreadcore08:57:271
179940,0migration/111654-21kthreadcore10:17:261
169940,0watchdog/131572-21if_err_eth107:12:251
169940,0watchdog/128631-21if_err_eth009:07:241
169940,0watchdog/126154-21ssh09:03:241
169940,0watchdog/122001-21grep07:27:291
169940,0watchdog/121741-21kthreadcore10:42:281
169940,0watchdog/12035-21ssh08:49:001
169940,0watchdog/119009-21grep10:02:241
169940,0watchdog/118418-21python07:47:331
169940,0watchdog/112205-21expr07:22:251
169940,0watchdog/10-21swapper/112:12:321
169940,0watchdog/10-21swapper/111:42:321
169940,0watchdog/10-21swapper/110:22:281
169940,0watchdog/10-21swapper/110:07:241
169940,0watchdog/10-21swapper/109:47:281
169940,0watchdog/10-21swapper/109:42:241
169940,0watchdog/10-21swapper/109:22:281
149940,0migration/07456-21kthreadcore08:52:280
149940,0migration/029232-21kthreadcore11:07:270
149940,0migration/028292-21kthreadcore11:27:260
149940,0migration/023160-21kthreadcore09:22:290
149940,0migration/021071-21runrttasks08:41:430
149940,0migration/01381-21irqcore08:02:300
339930,0migration/322476-21kthreadcore08:17:273
29715030,0irq/26-eth0-rx-0-21swapper/308:42:213
29715030,0irq/26-eth0-rx-0-21swapper/308:07:213
29715030,0irq/26-eth0-rx-0-21swapper/307:43:333
29715030,0irq/26-eth0-rx-0-21swapper/307:12:223
295389933,0cyclictest0-21swapper/208:04:342
295389932,0cyclictest3556-21lldpd08:42:282
295389932,0cyclictest25014-21awk07:07:272
295389930,0cyclictest0-21swapper/207:42:232
295379932,0cyclictest3556-21lldpd07:33:571
295379932,0cyclictest19697-21tr07:52:221
295379932,0cyclictest13515-21idleruntime-cro08:12:111
295379932,0cyclictest12413-21awk09:17:241
295379931,0cyclictest9145-21kthreadcore07:42:281
295369933,0cyclictest20178-21kworker/0:212:12:210
295369933,0cyclictest20178-21kworker/0:212:07:270
295369933,0cyclictest20178-21kworker/0:211:22:210
295369933,0cyclictest20178-21kworker/0:209:42:190
295369933,0cyclictest20178-21kworker/0:209:32:110
295369933,0cyclictest17843-21kworker/0:107:32:260
295369933,0cyclictest17843-21kworker/0:107:17:260
295369932,0cyclictest3514-21snmpd09:02:130
295369932,0cyclictest13310-21grep07:47:240
295369930,0cyclictest25071-21python07:52:320
249930,0watchdog/20-21swapper/208:32:242
179930,0migration/115953-21aten_r1power_vo11:22:221
169930,0watchdog/129091-21ssh12:08:521
169930,0watchdog/10-21swapper/108:17:321
169930,0watchdog/10-21swapper/106:52:291
339920,0migration/312730-21iostat_ios07:22:273
29715020,0irq/26-eth0-rx-0-21swapper/307:55:543
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional