You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 13:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Jan 01, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450610,0irq/25-eth10-21swapper/307:06:153
2066723826,0sleep20-21swapper/207:03:422
2091823524,0sleep10-21swapper/107:06:371
2066522723,0sleep00-21swapper/007:03:400
339960,0migration/37475-21sh10:14:493
339960,0migration/36693-21sh12:11:183
339960,0migration/331463-21munin-node10:28:313
339960,0migration/330088-21sh09:49:213
339960,0migration/329731-21sh12:04:043
339960,0migration/321284-21sh11:58:583
339960,0migration/318368-21sh11:00:063
339960,0migration/311881-21latency_hist08:38:113
29715060,0irq/26-eth0-rx-7094-21diskstats09:38:233
29715060,0irq/26-eth0-rx-18904-21aten2.4-expect10:43:233
29715060,0irq/26-eth0-rx-13801-21python09:58:363
29715060,0irq/26-eth0-rx-13116-21diskstats11:18:233
29715060,0irq/26-eth0-rx-0-21swapper/311:33:423
249960,0watchdog/20-21swapper/211:33:252
211299960,0cyclictest0-21swapper/211:21:142
211289965,0cyclictest0-21swapper/110:04:381
149960,0migration/08756-21sh12:13:250
149960,0migration/07024-21sh10:13:440
149960,0migration/04317-21sh11:31:060
149960,0migration/030265-21sh09:49:470
149960,0migration/02682-21sh10:52:480
149960,0migration/025874-21munin-node09:08:340
149960,0migration/021985-21latency07:53:260
149960,0migration/021676-21sh11:42:290
149960,0migration/019628-21latency_hist09:08:110
149960,0migration/017379-21sh10:40:320
149960,0migration/016478-21gltestperf07:28:280
149960,0migration/013799-21sh09:58:350
339950,0migration/37850-21kthreadcore11:13:293
339950,0migration/34238-21latency_hist09:18:113
339950,0migration/332313-21cpuspeed_turbos08:53:233
339950,0migration/330427-21taskset08:24:003
339950,0migration/329376-21taskset07:58:423
339950,0migration/326544-21munin-node08:48:283
339950,0migration/321448-21munin-node11:23:243
339950,0migration/320616-21missed_timers09:43:283
339950,0migration/319727-21latency_hist09:08:113
339950,0migration/317648-21munin-node10:03:263
339950,0migration/316705-21latency_hist07:53:113
339950,0migration/316567-21munin-node12:18:233
339950,0migration/316518-21hwlatdetect07:28:283
339950,0migration/315512-21munin-node07:48:303
339950,0migration/315456-21taskset12:34:303
339950,0migration/31086-21kthreadcore11:28:283
29725050,0irq/27-eth0-tx-7780-21munin-node12:13:232
29725050,0irq/27-eth0-tx-7385-21ssh12:30:122
29725050,0irq/27-eth0-tx-4332-21munin-node11:13:302
29725050,0irq/27-eth0-tx-30440-21aten_r1power_fr09:33:222
29725050,0irq/27-eth0-tx-29039-21munin-node07:38:302
29725050,0irq/27-eth0-tx-28012-21munin-node07:13:282
29725050,0irq/27-eth0-tx-195141diskmemload09:44:502
29725050,0irq/27-eth0-tx-16159-21munin-node12:18:212
29725050,0irq/27-eth0-tx-0-21swapper/211:51:412
29725050,0irq/27-eth0-tx-0-21swapper/211:42:242
29725050,0irq/27-eth0-tx-0-21swapper/211:02:532
29725050,0irq/27-eth0-tx-0-21swapper/210:30:422
29725050,0irq/27-eth0-tx-0-21swapper/210:24:342
29725050,0irq/27-eth0-tx-0-21swapper/209:28:362
29725050,0irq/27-eth0-tx-0-21swapper/209:16:002
29715050,0irq/26-eth0-rx-5589-21kthreadcore12:28:293
29715050,0irq/26-eth0-rx-3197-17auditd08:13:113
29715050,0irq/26-eth0-rx-29902-21ssh09:32:523
29715050,0irq/26-eth0-rx-1640-21latency_hist10:33:113
29715050,0irq/26-eth0-rx-13793-21fschecks_time09:23:243
29715050,0irq/26-eth0-rx-0-21swapper/312:16:473
29715050,0irq/26-eth0-rx-0-21swapper/311:53:513
29715050,0irq/26-eth0-rx-0-21swapper/311:39:313
29715050,0irq/26-eth0-rx-0-21swapper/310:39:513
29715050,0irq/26-eth0-rx-0-21swapper/310:23:193
29715050,0irq/26-eth0-rx-0-21swapper/310:12:343
29715050,0irq/26-eth0-rx-0-21swapper/309:54:403
29715050,0irq/26-eth0-rx-0-21swapper/309:35:123
29715050,0irq/26-eth0-rx-0-21swapper/309:14:343
29715050,0irq/26-eth0-rx-0-21swapper/308:38:293
259950,0migration/29401-21sh10:53:422
259950,0migration/28534-21taskset10:17:222
259950,0migration/24694-21kthreadcore07:43:272
259950,0migration/232472-21gltestperf12:08:262
259950,0migration/229552-21timerandwakeup09:48:362
259950,0migration/226131-21sh09:09:282
259950,0migration/218494-21munin-node08:18:242
259950,0migration/216685-21sh10:38:522
259950,0migration/215491-21sh10:18:522
259950,0migration/212613-21timerwakeupswit11:53:342
259950,0migration/210255-21kthreadcore08:58:292
259950,0migration/210126-21sh12:33:282
249950,0watchdog/25180-21if_err_eth108:08:252
249950,0watchdog/218887-21cut07:53:252
211299955,0cyclictest0-21swapper/212:01:262
211299955,0cyclictest0-21swapper/209:58:372
211299955,0cyclictest0-21swapper/209:26:082
211299953,0cyclictest0-21swapper/207:33:272
211299950,0cyclictest0-21swapper/212:26:492
211299950,0cyclictest0-21swapper/210:49:422
211299950,0cyclictest0-21swapper/210:11:522
211289955,0cyclictest0-21swapper/112:04:281
211289955,0cyclictest0-21swapper/109:13:341
211289950,0cyclictest0-21swapper/110:11:061
211279955,0cyclictest0-21swapper/011:48:540
211279955,0cyclictest0-21swapper/009:44:590
211279950,0cyclictest0-21swapper/012:18:130
211279950,0cyclictest0-21swapper/009:35:220
211279950,0cyclictest0-21swapper/007:38:130
179950,0migration/131159-21munin-node09:33:251
179950,0migration/13015-21timerwakeupswit11:28:341
179950,0migration/112351-21sh11:35:121
179950,0migration/110945-21sh11:14:391
169950,0watchdog/131691-21ssh12:25:561
149950,0migration/09972-21latency_hist07:48:110
149950,0migration/07790-21kthreadcore08:33:290
149950,0migration/06246-21kthreadcore07:18:280
149950,0migration/05854-21kthreadcore08:08:270
149950,0migration/04697-21kthreadcore08:53:290
149950,0migration/032690-21kthreadcore07:13:280
149950,0migration/03266-21kthreadcore12:28:280
149950,0migration/031113-21sh10:09:040
149950,0migration/030685-21kthreadcore10:28:300
149950,0migration/029770-21sh12:04:100
149950,0migration/029178-21timerwakeupswit07:58:340
149950,0migration/029083-21sh11:45:140
149950,0migration/028026-21sh09:13:160
149950,0migration/02761-21idleruntime10:33:240
149950,0migration/022318-21if_eth1.307:08:260
149950,0migration/018757-21kthreadcore11:58:290
149950,0migration/017576-21kthreadcore11:18:280
149950,0migration/017395-21munin-node10:03:260
149950,0migration/017274-21sh10:23:090
149950,0migration/017142-21taskset08:14:210
149950,0migration/014495-21kthreadcore09:23:280
149950,0migration/013877-21kthreadcore08:38:270
149950,0migration/01220-21kthreadcore08:28:290
149950,0migration/010928-21kthreadcore11:53:280
8215040,0irq/24-0000:00:0-21swapper/207:28:112
339940,0migration/3967-21kthreadcore11:48:303
339940,0migration/35420-21kthreadcore07:18:283
339940,0migration/323898-21taskset08:20:283
339940,0migration/32163-21kthreadcore08:03:283
339940,0migration/320742-21kthreadcore08:43:273
339940,0migration/312708-21kthreadcore07:23:293
339940,0migration/31168-21kthreadcore07:38:283
339940,0migration/31165-21kthreadcore08:28:293
29725040,0irq/27-eth0-tx-20982-21ssh11:23:142
29725040,0irq/27-eth0-tx-11461-21ssh09:20:272
29725040,0irq/27-eth0-tx-0-21swapper/212:03:372
29725040,0irq/27-eth0-tx-0-21swapper/211:43:262
29725040,0irq/27-eth0-tx-0-21swapper/211:28:242
29725040,0irq/27-eth0-tx-0-21swapper/211:08:112
29725040,0irq/27-eth0-tx-0-21swapper/210:43:242
29725040,0irq/27-eth0-tx-0-21swapper/210:33:122
29725040,0irq/27-eth0-tx-0-21swapper/210:04:042
29725040,0irq/27-eth0-tx-0-21swapper/209:53:232
29715040,0irq/26-eth0-rx-28048-21munin-node10:48:263
29715040,0irq/26-eth0-rx-0-21swapper/312:24:063
29715040,0irq/26-eth0-rx-0-21swapper/311:43:113
29715040,0irq/26-eth0-rx-0-21swapper/311:09:063
29715040,0irq/26-eth0-rx-0-21swapper/311:04:423
29715040,0irq/26-eth0-rx-0-21swapper/310:53:213
29715040,0irq/26-eth0-rx-0-21swapper/310:19:113
29715040,0irq/26-eth0-rx-0-21swapper/307:08:283
259940,0migration/2349-21hddtemp_smartct08:53:252
259940,0migration/231592-21fschecks_time08:28:262
259940,0migration/224365-21runrttasks08:23:162
259940,0migration/211693-21kthreadcore07:23:292
259940,0migration/21067-21kthreadcore08:03:282
249940,0watchdog/27858-21kthreadcore08:33:292
249940,0watchdog/226248-21ssh11:03:572
211299943,0cyclictest3514-21snmpd09:03:142
211299943,0cyclictest3514-21snmpd08:49:142
211299943,0cyclictest3514-21snmpd08:46:142
211299943,0cyclictest3514-21snmpd07:09:152
211299943,0cyclictest24328-21awk07:58:262
211299942,0cyclictest7832-21sed09:38:272
211299940,0cyclictest0-21swapper/207:18:552
211289943,0cyclictest9477-21fschecks_time12:33:251
211289943,0cyclictest7514-21awk11:53:251
211289943,0cyclictest3556-21lldpd07:10:311
211289943,0cyclictest3514-21snmpd10:38:151
211289943,0cyclictest3514-21snmpd08:39:141
211289943,0cyclictest3241-21irqbalance10:48:391
211289943,0cyclictest3241-21irqbalance09:53:591
211289943,0cyclictest3137-21awk07:18:251
211289943,0cyclictest22637-21awk09:28:271
211289943,0cyclictest19676-21awk10:43:271
211289943,0cyclictest19566-21awk08:43:241
211289943,0cyclictest17169-21awk12:18:261
211289943,0cyclictest15968-21awk11:58:271
211289940,0cyclictest0-21swapper/107:43:511
211279944,0cyclictest0-21swapper/011:10:350
211279943,0cyclictest436-21awk09:53:260
211279943,0cyclictest3556-21lldpd09:19:320
211279943,0cyclictest3514-21snmpd11:13:300
211279943,0cyclictest3241-21irqbalance10:47:590
211279943,0cyclictest23256-21awk07:33:260
179940,0migration/15062-21missed_timers08:53:291
179940,0migration/132349-21fschecks_time12:08:251
179940,0migration/131642-21fschecks_time08:28:261
179940,0migration/128544-21cpuspeed_turbos11:08:221
179940,0migration/128234-21cpuspeed_turbos07:13:231
179940,0migration/126612-21kthreadcore07:33:291
179940,0migration/126601-21fschecks_time10:28:251
179940,0migration/123975-21kthreadcore11:43:271
179940,0migration/12367-21kthreadcore10:13:271
179940,0migration/123317-21latency_hist07:58:111
179940,0migration/120664-21kthreadcore07:28:321
179940,0migration/118510-21munin-node10:23:261
179940,0migration/114907-21turbostat09:43:111
179940,0migration/114691-21sh12:15:201
179940,0migration/114316-21kthreadcore08:13:291
179940,0migration/113914-21aten_r1power_cu11:38:221
179940,0migration/113725-21kthreadcore10:18:321
179940,0migration/112597-21kthreadcore07:48:271
179940,0migration/112089-21kthreadcore07:23:291
179940,0migration/111490-21taskset09:20:301
169940,0watchdog/16947-21ssh12:29:081
169940,0watchdog/15378-21ssh11:50:521
169940,0watchdog/14006-21ssh10:53:251
169940,0watchdog/127962-21ssh11:08:051
169940,0watchdog/126414-21ssh09:10:171
169940,0watchdog/124206-21kthreadcore11:23:291
169940,0watchdog/123991-21fschecks_count09:48:251
169940,0watchdog/114603-21ssh09:59:411
169940,0watchdog/111964-21df_inode10:58:251
169940,0watchdog/10-21swapper/109:45:491
169940,0watchdog/10-21swapper/108:23:291
149940,0migration/09237-21kthreadcore11:33:290
149940,0migration/032238-21runrttasks12:27:160
149940,0migration/022592-21kthreadcore08:18:270
149940,0migration/017590-21aten_r1power_cu10:23:220
29725030,0irq/27-eth0-tx-0-21swapper/208:13:222
29715030,0irq/26-eth0-rx-0-21swapper/309:05:143
29715030,0irq/26-eth0-rx-0-21swapper/308:58:303
29715030,0irq/26-eth0-rx-0-21swapper/308:17:533
29715030,0irq/26-eth0-rx-0-21swapper/307:43:303
29715030,0irq/26-eth0-rx-0-21swapper/307:13:223
211299932,0cyclictest3514-21snmpd08:38:142
211299932,0cyclictest12011-21cut07:48:272
211289932,0cyclictest5166-21awk08:08:251
211289932,0cyclictest3295-21sensors08:03:331
211289932,0cyclictest30479-21kthreadcore07:38:271
211289932,0cyclictest2846-21awk10:33:251
211289932,0cyclictest26213-21awk08:48:261
211289932,0cyclictest13990-21awk09:03:251
211289932,0cyclictest13613-21awk11:18:251
211279933,0cyclictest3556-21lldpd09:29:330
211279932,0cyclictest3556-21lldpd10:56:020
211279932,0cyclictest3514-21snmpd07:46:000
211279932,0cyclictest32494-21tail12:08:260
211279932,0cyclictest3241-21irqbalance12:33:280
211279932,0cyclictest25240-21awk08:23:260
211279932,0cyclictest20241-21tr11:03:220
211279932,0cyclictest19645-21awk08:43:250
211279932,0cyclictest19514-21diskmemload11:23:460
211279932,0cyclictest17368-21sensors10:58:350
211279932,0cyclictest14008-21grep07:23:290
211279932,0cyclictest1140-21kthreadcore08:03:280
211279930,0cyclictest13133-21diskmemload09:38:580
169930,0watchdog/10-21swapper/109:23:531
169930,0watchdog/10-21swapper/108:58:291
169930,0watchdog/10-21swapper/108:33:291
169930,0watchdog/10-21swapper/107:53:251
29715020,0irq/26-eth0-rx-0-21swapper/307:33:323
211279921,0cyclictest6784-21df_abs08:58:240
211279921,0cyclictest25375-21munin-node08:48:230
169920,0watchdog/10-21swapper/108:18:171
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional