You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 23:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Dec 23, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2491124130,0sleep00-21swapper/007:05:560
1880723725,0sleep10-21swapper/107:04:081
2498423324,0sleep20-21swapper/207:06:462
304450320,0irq/25-eth10-21swapper/307:08:233
339960,0migration/325996-21sh11:40:593
339960,0migration/324131-21sh11:19:483
339960,0migration/322729-21sh10:43:113
339960,0migration/313721-21fschecks_time07:24:143
339960,0migration/313037-21sh12:13:293
29725060,0irq/27-eth0-tx-20719-21munin-node07:54:240
29715060,0irq/26-eth0-rx-5326-21port_sshpriv09:14:203
29715060,0irq/26-eth0-rx-3765-21sh10:11:293
29715060,0irq/26-eth0-rx-24032-21perl10:44:133
29715060,0irq/26-eth0-rx-20859-21sed12:34:243
29715060,0irq/26-eth0-rx-13442-21cut11:54:173
29715060,0irq/26-eth0-rx-11483-21gltestperf11:34:183
252499960,0cyclictest0-21swapper/110:31:221
252489965,0cyclictest25404-21sh09:44:410
149960,0migration/05943-21hwlatdetect10:14:150
8215050,0irq/24-0000:00:377-21munin-plugin-st07:39:022
339950,0migration/39425-21switchtime08:54:243
339950,0migration/37442-21diskmemload11:10:143
339950,0migration/332140-21kthreadcore07:59:183
339950,0migration/332034-21taskset11:06:523
339950,0migration/330367-21kthreadcore09:49:173
339950,0migration/330304-21munin-node08:49:183
339950,0migration/329423-21taskset10:27:273
339950,0migration/327486-21timerandwakeup08:19:253
339950,0migration/322685-21aten_r1power_cu08:44:133
339950,0migration/320760-21aten2.4-expect11:59:133
339950,0migration/320680-21munin-node08:14:223
339950,0migration/317146-21kthreadcore10:19:203
339950,0migration/316821-21df_abs10:59:143
339950,0migration/313235-21kthreadcore09:39:173
29725050,0irq/27-eth0-tx-8940-21fschecks_count09:19:142
29725050,0irq/27-eth0-tx-7508-21kthreadcore11:29:192
29725050,0irq/27-eth0-tx-7508-21kthreadcore11:29:192
29725050,0irq/27-eth0-tx-3735-21munin-node09:54:192
29725050,0irq/27-eth0-tx-315952chrt11:05:502
29725050,0irq/27-eth0-tx-29069-21munin-node10:09:132
29725050,0irq/27-eth0-tx-1775-21cut12:24:182
29725050,0irq/27-eth0-tx-1703-40yum-updatesd-he09:50:102
29725050,0irq/27-eth0-tx-14923-21munin-node10:39:122
29725050,0irq/27-eth0-tx-14063-21munin-node07:49:180
29725050,0irq/27-eth0-tx-12195-21munin-node09:59:182
29725050,0irq/27-eth0-tx-0-21swapper/211:02:432
29725050,0irq/27-eth0-tx-0-21swapper/210:54:232
29725050,0irq/27-eth0-tx-0-21swapper/210:19:012
29725050,0irq/27-eth0-tx-0-21swapper/209:29:502
29725050,0irq/27-eth0-tx-0-21swapper/008:59:050
29725050,0irq/27-eth0-tx-0-21swapper/008:19:140
29725050,0irq/27-eth0-tx-0-21swapper/008:06:130
29715050,0irq/26-eth0-rx-9285-21runrttasks11:31:193
29715050,0irq/26-eth0-rx-9285-21runrttasks11:31:193
29715050,0irq/26-eth0-rx-6096-21kthreadcore09:54:193
29715050,0irq/26-eth0-rx-330-21cut07:14:143
29715050,0irq/26-eth0-rx-30325-21kthreadcore07:09:223
29715050,0irq/26-eth0-rx-26815-21fschecks_time11:24:143
29715050,0irq/26-eth0-rx-13537-21kthreadcore09:59:183
29715050,0irq/26-eth0-rx-0-21swapper/312:29:193
29715050,0irq/26-eth0-rx-0-21swapper/312:28:453
29715050,0irq/26-eth0-rx-0-21swapper/312:14:483
29715050,0irq/26-eth0-rx-0-21swapper/312:07:133
29715050,0irq/26-eth0-rx-0-21swapper/311:46:353
29715050,0irq/26-eth0-rx-0-21swapper/311:14:173
29715050,0irq/26-eth0-rx-0-21swapper/310:50:253
29715050,0irq/26-eth0-rx-0-21swapper/310:38:243
29715050,0irq/26-eth0-rx-0-21swapper/310:04:083
29715050,0irq/26-eth0-rx-0-21swapper/309:44:153
29715050,0irq/26-eth0-rx-0-21swapper/309:34:193
29715050,0irq/26-eth0-rx-0-21swapper/309:24:103
29715050,0irq/26-eth0-rx-0-21swapper/309:19:033
29715050,0irq/26-eth0-rx-0-21swapper/309:09:243
29715050,0irq/26-eth0-rx-0-21swapper/308:04:193
252509950,0cyclictest0-21swapper/212:35:462
252509950,0cyclictest0-21swapper/210:38:382
252509950,0cyclictest0-21swapper/210:29:392
252499955,0cyclictest0-21swapper/112:04:131
252499955,0cyclictest0-21swapper/111:45:181
252499955,0cyclictest0-21swapper/110:40:211
252499955,0cyclictest0-21swapper/109:11:451
252499950,0cyclictest0-21swapper/112:35:071
252489955,0cyclictest0-21swapper/011:04:550
252489955,0cyclictest0-21swapper/008:24:260
252489952,0cyclictest11913-21munin-run11:54:010
252489950,0cyclictest0-21swapper/009:38:030
252489950,0cyclictest0-21swapper/009:19:080
259950,0migration/25443-21timerandwakeup10:49:272
259950,0migration/229648-21fschecks_time12:04:142
259950,0migration/22493-21fschecks_count09:34:162
259950,0migration/213440-21latency_hist12:14:002
259950,0migration/211583-21sh11:53:182
259950,0migration/211101-21munin-node11:34:172
259950,0migration/210744-21aten_r1power_po09:39:122
249950,0watchdog/226401-21kthreadcore10:24:192
249950,0watchdog/223567-21cut11:39:192
19150,0ktimersoftd/10-21swapper/109:20:441
179950,0migration/15179-21aten_r1power_en10:14:121
179950,0migration/13395-21kthreadcore08:04:191
179950,0migration/131707-21timerandwakeup09:29:251
179950,0migration/124699-21kthreadcore07:29:211
179950,0migration/120337-21sh12:15:521
179950,0migration/11557-21sh11:28:091
179950,0migration/112619-21taskset12:12:301
179950,0migration/112412-21kthreadcore08:59:191
179950,0migration/11078-21fschecks_count07:39:141
169950,0watchdog/131421-21tr08:24:201
169950,0watchdog/120192-21tr08:39:201
169950,0watchdog/110516-21grep07:19:201
169950,0watchdog/10-21swapper/110:04:151
149950,0migration/09988-21kthreadcore07:19:200
149950,0migration/05611-21kthreadcore11:09:180
149950,0migration/05598-21kthreadcore08:29:180
149950,0migration/05373-21cpuspeed_turbos12:09:130
149950,0migration/032474-21kthreadcore10:29:170
149950,0migration/031834-21aten_r1power_cu10:49:150
149950,0migration/031017-21kthreadcore11:24:180
149950,0migration/030655-21kthreadcore11:44:190
149950,0migration/029223-21switchtime10:44:230
149950,0migration/028227-21kthreadcore07:34:190
149950,0migration/027880-21taskset10:06:550
149950,0migration/027859-21switchtime12:19:210
149950,0migration/027306-21kthreadcore07:09:210
149950,0migration/026576-21kthreadcore10:24:190
149950,0migration/024769-21kthreadcore07:29:210
149950,0migration/023735-21kthreadcore11:59:180
149950,0migration/020222-21kthreadcore09:04:190
149950,0migration/018757-21kthreadcore10:19:200
149950,0migration/015708-21kthreadcore07:24:170
149950,0migration/013978-21sh10:37:280
149950,0migration/013329-21irqcore11:54:160
149950,0migration/010772-21aten_r1power_cu11:34:150
8215040,0irq/24-0000:00:0-21swapper/211:19:012
339940,0migration/37123-21taskset07:43:133
339940,0migration/36537-21aten_r1power_po07:19:143
339940,0migration/332588-21kthreadcore08:24:203
339940,0migration/331419-21diskmemload09:29:233
339940,0migration/315707-21kthreadcore07:49:173
339940,0migration/314536-21taskset08:10:563
339940,0migration/311454-21kthreadcore10:54:203
339940,0migration/310888-21kthreadcore08:34:193
339940,0migration/310244-21latency_hist08:59:023
29725040,0irq/27-eth0-tx-5129-21munin-node10:14:142
29725040,0irq/27-eth0-tx-3635-21munin-node08:54:240
29725040,0irq/27-eth0-tx-23427-21diskmemload12:29:512
29725040,0irq/27-eth0-tx-22805-21ssh12:19:132
29725040,0irq/27-eth0-tx-20931-21df11:59:132
29725040,0irq/27-eth0-tx-19070-21ssh09:44:052
29725040,0irq/27-eth0-tx-15134-21telnet08:14:120
29725040,0irq/27-eth0-tx-0-21swapper/212:10:312
29725040,0irq/27-eth0-tx-0-21swapper/211:25:512
29725040,0irq/27-eth0-tx-0-21swapper/211:15:202
29725040,0irq/27-eth0-tx-0-21swapper/211:09:082
29725040,0irq/27-eth0-tx-0-21swapper/209:24:112
29725040,0irq/27-eth0-tx-0-21swapper/209:14:152
29725040,0irq/27-eth0-tx-0-21swapper/209:10:582
29725040,0irq/27-eth0-tx-0-21swapper/009:09:020
29725040,0irq/27-eth0-tx-0-21swapper/007:59:160
29715040,0irq/26-eth0-rx-17063-21chrt09:04:133
29715040,0irq/26-eth0-rx-0-21swapper/312:19:333
29715040,0irq/26-eth0-rx-0-21swapper/311:49:173
29715040,0irq/26-eth0-rx-0-21swapper/310:29:123
29715040,0irq/26-eth0-rx-0-21swapper/310:16:353
29715040,0irq/26-eth0-rx-0-21swapper/307:44:193
252509943,0cyclictest576-21awk07:14:152
252509943,0cyclictest3514-21snmpd08:56:252
252509943,0cyclictest3514-21snmpd08:40:252
252509943,0cyclictest3514-21snmpd08:38:252
252509943,0cyclictest3514-21snmpd08:31:252
252509943,0cyclictest3514-21snmpd08:03:252
252509943,0cyclictest3514-21snmpd07:25:252
252509943,0cyclictest15973-21awk08:14:162
252499943,0cyclictest3556-21lldpd11:18:421
252499943,0cyclictest3538-21awk08:29:161
252499943,0cyclictest3142-21awk11:29:161
252499943,0cyclictest3142-21awk11:29:161
252499943,0cyclictest28221-21awk07:59:151
252499943,0cyclictest22833-21awk10:24:171
252499943,0cyclictest21486-21awk11:59:161
252499943,0cyclictest13131-21awk09:59:161
252499943,0cyclictest13091-21awk11:54:151
252499940,0cyclictest0-21swapper/111:50:111
252489944,0cyclictest23427-21diskmemload09:18:210
252489943,0cyclictest602-21awk07:14:160
252489943,0cyclictest3514-21snmpd09:54:100
252489943,0cyclictest30177-21awk08:49:170
252489943,0cyclictest30071-21awk10:09:160
252489943,0cyclictest28466-21awk09:49:150
252489943,0cyclictest1563-21awk07:39:160
252489943,0cyclictest10247-21awk11:14:150
252489940,0cyclictest0-21swapper/010:59:050
259940,0migration/26403-21munin-node07:39:222
259940,0migration/231762-21kthreadcore08:24:202
259940,0migration/228094-21ntp_kernel_pll_08:44:212
259940,0migration/227076-21kthreadcore07:09:212
259940,0migration/225850-21kthreadcore08:19:202
259940,0migration/224106-21gltestperf10:44:142
259940,0migration/224088-21kthreadcore07:54:192
259940,0migration/220175-21fschecks_count07:29:162
259940,0migration/215867-21kthreadcore07:49:172
259940,0migration/211573-21irqrtprio08:59:192
249940,0watchdog/28097-21fschecks_time07:44:162
249940,0watchdog/27062-21perl07:19:162
249940,0watchdog/232406-21kthreadcore08:49:202
249940,0watchdog/228179-21perl11:44:152
249940,0watchdog/218107-21wc11:54:232
249940,0watchdog/20-21swapper/210:04:152
249940,0watchdog/20-21swapper/209:04:202
249940,0watchdog/20-21swapper/208:09:162
249940,0watchdog/20-21swapper/208:04:202
179940,0migration/18591-21cpuspeed_turbos08:09:141
179940,0migration/16539-21aten_r1power_fr12:29:131
179940,0migration/16468-21kthreadcore09:54:191
179940,0migration/132293-21kthreadcore09:49:181
179940,0migration/130767-21munin-node07:09:221
179940,0migration/126446-21aten2.4-expect07:34:131
179940,0migration/126361-21kthreadcore08:19:201
179940,0migration/124209-21kthreadcore08:44:181
179940,0migration/12394-21df09:34:151
179940,0migration/122485-21kthreadcore07:54:181
179940,0migration/12117-21kthreadcore07:14:171
179940,0migration/119766-21kthreadcore09:24:201
179940,0migration/114005-21kthreadcore11:34:221
179940,0migration/112890-21kthreadcore08:34:191
169940,0watchdog/17905-21ssh10:54:071
169940,0watchdog/170672chrt10:52:391
169940,0watchdog/16559-21runrttasks09:07:041
169940,0watchdog/128478-21taskset12:20:111
169940,0watchdog/123568-21tr11:39:191
169940,0watchdog/123427-21diskmemload11:09:031
169940,0watchdog/119057-21ssh09:44:041
169940,0watchdog/117396-21ssh09:40:441
169940,0watchdog/117181-21gltestperf10:59:151
169940,0watchdog/113082-21python07:44:241
169940,0watchdog/10-21swapper/111:06:271
169940,0watchdog/10-21swapper/110:34:151
169940,0watchdog/10-21swapper/110:19:191
169940,0watchdog/10-21swapper/110:09:151
169940,0watchdog/10-21swapper/108:49:161
169940,0watchdog/10-21swapper/107:49:201
149940,0migration/07137-21kthreadcore11:29:190
149940,0migration/07137-21kthreadcore11:29:190
149940,0migration/023981-21kthreadcore11:39:190
149940,0migration/019987-21kthreadcore09:24:200
149940,0migration/016176-21kthreadcore09:59:180
149940,0migration/010658-21runrttasks09:39:110
35130,0ktimersoftd/30-21swapper/307:36:403
29725030,0irq/27-eth0-tx-0-21swapper/008:44:150
29715030,0irq/26-eth0-rx-0-21swapper/308:39:133
29715030,0irq/26-eth0-rx-0-21swapper/307:56:523
29715030,0irq/26-eth0-rx-0-21swapper/307:29:153
252499933,0cyclictest15948-21awk08:14:161
252499932,0cyclictest24331-21awk10:44:151
252499932,0cyclictest13935-21awk07:24:151
252489933,0cyclictest15793-21awk12:34:160
252489933,0cyclictest0-21swapper/012:24:330
252489932,0cyclictest9206-21awk08:09:170
252489932,0cyclictest29412-21df_abs12:04:130
252489932,0cyclictest26817-21awk09:29:180
252489932,0cyclictest23595-21sensors11:19:230
252489932,0cyclictest16699-21grep08:39:150
252489932,0cyclictest15350-21forks10:39:130
252489932,0cyclictest14437-21awk12:14:160
252489932,0cyclictest13076-21sensors07:44:240
252489932,0cyclictest12916-21sh12:30:000
179930,0migration/18317-21kthreadcore08:54:191
169930,0watchdog/122446-21cut11:19:191
169930,0watchdog/10-21swapper/112:24:151
169930,0watchdog/10-21swapper/109:15:481
29715020,0irq/26-eth0-rx-0-21swapper/308:29:133
252489920,0cyclictest9280-21wc08:34:020
252489920,0cyclictest0-21swapper/010:54:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional