You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 00:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Sep 22, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2329599735,52cyclictest9529-21ssh09:10:141
2328799583,43cyclictest13090-21ssh11:00:120
23303995628,21cyclictest27-21ksoftirqd/209:55:142
2328799521,7cyclictest131rcu_preempt11:55:120
23295995132,14cyclictest12964-21ssh12:30:111
2328799512,39cyclictest9547-21ssh10:40:100
2328799512,39cyclictest18552-21ssh09:15:140
23303994917,19cyclictest24222-21ssh10:30:172
23295994832,11cyclictest31328-21apt-get09:40:111
23315994628,9cyclictest35966irq/58-eth012:05:143
23315994628,9cyclictest35966irq/58-eth012:05:133
23303994616,13cyclictest27-21ksoftirqd/211:45:142
2328799463,3cyclictest131rcu_preempt11:15:120
2328799461,34cyclictest22103-21ssh11:05:130
23315994321,12cyclictest35966irq/58-eth011:15:133
2330399424,20cyclictest16659-21ssh11:20:122
2329599425,26cyclictest21-21ksoftirqd/108:10:151
2329599421,33cyclictest27619-21ssh10:50:171
2328799421,33cyclictest20615-21ssh10:10:160
2331599393,24cyclictest35966irq/58-eth012:00:133
2331599382,24cyclictest35966irq/58-eth011:20:133
2330399388,9cyclictest27-21ksoftirqd/208:05:152
2330399387,20cyclictest27-21ksoftirqd/207:35:132
23295993831,3cyclictest29550-21apt-get07:50:131
23295993830,4cyclictest14673-21apt-get11:55:121
23295993824,9cyclictest23442-21apt-get07:10:141
2328799382,8cyclictest131rcu_preempt10:20:130
23295993713,10cyclictest227431/usr/sbin/munin08:50:181
23295993710,11cyclictest5661-21ssh10:35:451
2331599363,21cyclictest35966irq/58-eth011:30:133
23303993624,7cyclictest27-21ksoftirqd/207:50:132
2329599368,10cyclictest22523-21ssh11:21:421
23295993630,3cyclictest3957-21apt-get12:25:131
23295993622,11cyclictest22097-21apt-get11:05:131
2331599355,16cyclictest35966irq/58-eth011:25:123
2330399354,22cyclictest0-21swapper/209:30:152
23303993523,6cyclictest0-21swapper/211:30:122
23295993528,3cyclictest13279-21apt-get09:30:171
23295993511,10cyclictest4685-21sh11:49:141
23295993510,9cyclictest2422-21ssh09:23:221
23295993510,10cyclictest9866-21rm09:26:281
2330399342,16cyclictest9105-21diskmemload12:06:382
2330399342,16cyclictest9105-21diskmemload12:06:382
23295993427,4cyclictest7967-21apt-get09:45:131
23295993426,4cyclictest31079-21apt-get11:10:121
23295993425,6cyclictest9458-21ssh12:10:131
23295993423,8cyclictest7668-21apt-get11:15:121
23295993421,8cyclictest0-21swapper/110:30:181
23295993410,9cyclictest17889-21rm10:07:161
2329599340,25cyclictest0-21swapper/108:35:141
2329599340,25cyclictest0-21swapper/108:35:131
2328799341,26cyclictest11580-21ssh10:05:130
2331599333,20cyclictest35966irq/58-eth009:55:143
2331599331,15cyclictest0-21swapper/309:30:153
2330399333,23cyclictest0-21swapper/212:04:442
2330399332,20cyclictest13082-21rm11:00:122
2329599339,8cyclictest32711-21ssh09:57:431
23295993326,3cyclictest4042-21apt-get10:55:111
23295993322,8cyclictest18435-21ssh12:15:131
23295993319,9cyclictest27364-21apt-get12:20:131
23295993319,7cyclictest29666-21ssh10:15:131
23295993311,9cyclictest15626-21rm10:41:551
2329599329,8cyclictest6002-21tr10:00:231
2329599327,8cyclictest9105-21diskmemload11:35:231
23295993221,8cyclictest354-21apt-get12:05:141
23295993221,8cyclictest354-21apt-get12:05:141
2329599321,24cyclictest15122-21ssh10:25:131
2329599321,24cyclictest15122-21ssh10:25:131
2331599313,16cyclictest35966irq/58-eth010:05:133
2330399313,8cyclictest23277-21cyclictest07:45:142
23295993110,8cyclictest9815-21ssh11:33:551
2328799315,22cyclictest2176-21ssh11:30:130
2328799312,22cyclictest381-21ssh12:05:140
2328799312,22cyclictest381-21ssh12:05:140
2331599303,19cyclictest0-21swapper/310:49:363
2330399309,16cyclictest27-21ksoftirqd/212:15:132
2330399305,17cyclictest27346irq/40-dwc2_hso11:15:122
2330399301,23cyclictest0-21swapper/211:05:162
2329599308,10cyclictest20022-21ls07:40:121
2329599302,21cyclictest5799-21ssh11:50:171
23295993012,13cyclictest9810-21ntp_192.168.11410:20:261
23295993011,7cyclictest9105-21diskmemload09:19:251
2328799302,13cyclictest3984-21ssh12:25:130
2328799301,22cyclictest12996-21ssh12:30:130
2331599293,19cyclictest0-21swapper/309:44:383
23315992917,7cyclictest35966irq/58-eth011:55:133
2330399293,21cyclictest0-21swapper/208:02:392
2330399293,11cyclictest0-21swapper/208:45:382
2329599299,6cyclictest9105-21diskmemload10:10:561
2329599299,11cyclictest0-21swapper/107:55:171
2329599298,7cyclictest20068-21sh11:40:101
23295992911,8cyclictest29029-21ssh12:38:041
2328799292,22cyclictest28401-21ssh11:44:410
2331599281,16cyclictest35966irq/58-eth009:15:153
2330399283,16cyclictest27346irq/40-dwc2_hso11:10:122
2330399282,20cyclictest0-21swapper/208:55:142
2330399282,11cyclictest12097-21ssh10:59:222
2329599287,9cyclictest30598-21ssh12:02:501
23295992816,9cyclictest24969-21fw_packets07:45:171
23295992816,9cyclictest21398-21fw_packets08:15:161
23295992814,11cyclictest23404-21ssh09:52:271
23295992814,10cyclictest0-21swapper/108:25:131
23295992814,10cyclictest0-21swapper/108:25:121
2328799282,23cyclictest25671-21ssh11:25:120
2328799282,20cyclictest32095-21ssh12:04:380
2328799281,18cyclictest31088-21rm11:10:110
2331599272,7cyclictest0-21swapper/307:25:123
2330399273,16cyclictest27346irq/40-dwc2_hso08:40:132
2330399272,15cyclictest27346irq/40-dwc2_hso08:25:122
2330399272,15cyclictest27346irq/40-dwc2_hso08:25:122
2330399272,10cyclictest0-21swapper/209:25:132
23303992720,3cyclictest261rcuc/210:40:102
23295992715,9cyclictest27402-21entropy08:55:161
23295992714,10cyclictest17793-21df_inode08:45:151
23295992712,12cyclictest4837-21/usr/sbin/munin09:05:171
2328799273,17cyclictest13277-21ssh09:30:150
2328799272,19cyclictest26794-21ssh10:49:360
2331599263,19cyclictest35966irq/58-eth009:20:143
2331599262,18cyclictest35966irq/58-eth010:30:173
2331599262,16cyclictest35966irq/58-eth010:25:123
2331599262,16cyclictest35966irq/58-eth010:25:123
2330399264,14cyclictest27346irq/40-dwc2_hso07:25:132
2330399263,18cyclictest27346irq/40-dwc2_hso09:35:142
2330399262,9cyclictest23373-21ssh10:29:382
2330399262,9cyclictest23373-21ssh10:29:382
2330399262,16cyclictest27346irq/40-dwc2_hso09:00:142
2330399261,21cyclictest19775-21ssh11:40:012
2329599269,14cyclictest28406-21fschecks_time07:15:181
2329599262,18cyclictest22502-21ssh09:35:161
23295992613,9cyclictest32591-21irqcore09:00:191
23295992613,10cyclictest13453-21irqcore11:00:171
23315992514,8cyclictest35966irq/58-eth011:00:133
2330399256,11cyclictest27-21ksoftirqd/211:50:162
2330399253,7cyclictest0-21swapper/210:19:382
2330399252,9cyclictest0-21swapper/209:45:382
23303992512,6cyclictest27-21ksoftirqd/210:50:172
23303992512,11cyclictest0-21swapper/208:35:172
23303992512,11cyclictest0-21swapper/208:35:172
2330399251,12cyclictest16995-21ssh09:50:162
2329599258,12cyclictest21-21ksoftirqd/108:09:381
2329599252,10cyclictest0-21swapper/110:45:111
23295992516,6cyclictest26239-21fw_packets08:20:151
23295992513,9cyclictest6277-21grep08:30:221
23295992513,9cyclictest29368-21ntp_states11:25:271
23295992512,10cyclictest8737-21missed_timers07:25:221
23295992512,10cyclictest19731-21awk07:40:011
23295992512,10cyclictest17475-21apt08:45:001
2328799251,18cyclictest9105-21diskmemload09:35:140
2331599243,12cyclictest35966irq/58-eth012:35:133
2330399242,19cyclictest0-21swapper/212:10:152
2330399242,16cyclictest0-21swapper/209:20:042
2330399241,19cyclictest0-21swapper/207:20:132
2330399241,14cyclictest26868-21ntp_kernel_err07:10:242
2330399240,8cyclictest22214-21kthreadcore10:10:242
2329599247,13cyclictest1089-21sed07:20:171
23295992413,8cyclictest13745-21ntp_192.168.11407:30:231
23295992412,9cyclictest23314-21sort07:10:011
2331599233,13cyclictest0-21swapper/311:44:413
2331599233,13cyclictest0-21swapper/309:27:383
2331599232,14cyclictest35966irq/58-eth009:10:133
2330399235,9cyclictest24528-21cron07:45:012
2330399232,6cyclictest0-21swapper/211:43:382
2330399232,6cyclictest0-21swapper/209:43:382
2330399232,16cyclictest22030-21ssh12:35:132
2330399232,16cyclictest0-21swapper/210:49:392
2330399232,16cyclictest0-21swapper/207:30:382
2330399231,6cyclictest14871-21grep11:55:162
2328799231,4cyclictest30591-21munin-run08:25:010
2331599225,9cyclictest0-21swapper/309:50:163
2331599223,12cyclictest35966irq/58-eth010:10:173
2331599223,12cyclictest35966irq/58-eth008:55:133
23315992214,5cyclictest35966irq/58-eth012:25:133
23315992210,6cyclictest0-21swapper/310:54:193
2330399223,3cyclictest0-21swapper/209:10:132
2330399222,6cyclictest0-21swapper/210:24:132
2330399221,9cyclictest5832-21grep07:55:292
2330399221,11cyclictest1376-21kthreadcore10:35:182
23303992210,5cyclictest31001-21ssh12:20:252
2329599227,12cyclictest7303-21irqrtprio08:00:191
2328799229,10cyclictest25651-21ssh12:35:230
2328799222,18cyclictest3083-21ssh10:54:190
2328799221,16cyclictest9105-21diskmemload10:55:120
2328799221,16cyclictest32121-21find09:00:130
2331599213,14cyclictest35966irq/58-eth011:50:173
2331599213,10cyclictest35966irq/58-eth007:35:123
2331599212,13cyclictest35966irq/58-eth012:30:123
2331599211,13cyclictest35966irq/58-eth007:20:123
2330399211,18cyclictest2556-21acpi10:00:122
2330399211,17cyclictest25660-21apt-get11:25:142
2330399211,16cyclictest0-21swapper/209:05:222
2330399211,11cyclictest6351-21tr12:25:212
2330399211,11cyclictest22727-21idleruntime08:50:192
2330399211,10cyclictest22804-21kthreadcore08:15:202
23303992110,8cyclictest16910-21irqrtprio08:10:202
2328799214,7cyclictest12-21ksoftirqd/011:40:010
2328799212,17cyclictest7241-21ssh09:44:380
2328799212,16cyclictest29682-21ssh10:15:150
23287992114,4cyclictest12-21ksoftirqd/008:20:010
2328799211,3cyclictest0-21swapper/012:10:190
2328799211,17cyclictest23277-21cyclictest07:10:010
2328799211,11cyclictest4421-21tune2fs09:25:160
2331599202,13cyclictest36066irq/59-eth010:00:123
23315992010,7cyclictest35966irq/58-eth011:35:143
2330399205,5cyclictest27599-21kthreadcore08:20:182
2330399201,9cyclictest4028-21kthreadcore08:30:192
2330399201,9cyclictest23348-21latency_hist07:10:012
2330399201,16cyclictest19138-21irqrtprio09:15:202
2330399201,10cyclictest15484-21cut12:30:212
2330399201,10cyclictest12806-21ssh10:05:202
2330399200,7cyclictest28365-21fschecks_count07:15:192
2328799207,10cyclictest2070-21grep07:55:170
2328799206,11cyclictest16560-21cron09:50:010
2328799202,7cyclictest131rcu_preempt09:10:130
2328799202,3cyclictest18412-21apt-get12:15:120
2328799201,8cyclictest17479-21munin-run08:45:010
2328799201,6cyclictest7786-21munin-run08:35:010
2328799201,2cyclictest0-21swapper/011:20:140
2328799201,16cyclictest29788-21ls07:50:180
2328799201,12cyclictest5760-21find11:50:150
2331599199,7cyclictest0-21swapper/310:56:373
2331599199,6cyclictest0-21swapper/311:48:463
2331599198,8cyclictest0-21swapper/312:16:483
2331599192,9cyclictest0-21swapper/312:20:113
2331599192,10cyclictest0-21swapper/308:45:143
2331599191,13cyclictest0-21swapper/310:35:143
2331599191,13cyclictest0-21swapper/307:45:173
2331599191,12cyclictest0-21swapper/307:30:173
2331599191,11cyclictest0-21swapper/310:45:013
23315991910,6cyclictest35966irq/58-eth010:20:143
23315991910,6cyclictest35966irq/58-eth009:35:133
2328799199,8cyclictest31137-21memory09:20:240
2328799199,7cyclictest16947-21irqrtprio08:10:200
2328799196,4cyclictest131rcu_preempt08:00:190
2328799196,10cyclictest3577-21cron12:25:010
2328799192,13cyclictest605-21ssh10:35:120
2328799191,9cyclictest21566-21grep08:45:280
23287991912,4cyclictest29806-21cut11:45:190
23287991911,6cyclictest29056-21ssh10:30:360
2331599189,6cyclictest0-21swapper/312:10:223
2331599183,10cyclictest0-21swapper/309:49:323
2331599183,10cyclictest0-21swapper/309:00:383
2331599182,12cyclictest35966irq/58-eth011:05:153
2331599182,11cyclictest0-21swapper/308:00:383
2331599182,10cyclictest0-21swapper/308:40:143
2331599182,10cyclictest0-21swapper/308:30:153
2331599182,10cyclictest0-21swapper/308:25:143
2331599182,10cyclictest0-21swapper/308:25:143
2331599181,12cyclictest0-21swapper/308:15:133
23315991810,6cyclictest0-21swapper/310:15:113
2328799189,7cyclictest12-21ksoftirqd/007:35:010
2328799189,6cyclictest841-21wc07:20:160
2328799188,7cyclictest28317-21kthreadcore08:55:200
2328799186,8cyclictest12-21ksoftirqd/009:55:140
2328799186,7cyclictest8235-21timerandwakeup09:05:270
2328799183,13cyclictest16225-21grep10:25:190
2328799183,13cyclictest16225-21grep10:25:190
2328799181,8cyclictest16766systemd-network08:50:200
2331599172,10cyclictest0-21swapper/311:10:053
2331599172,10cyclictest0-21swapper/308:50:153
2331599172,10cyclictest0-21swapper/308:35:043
2331599172,10cyclictest0-21swapper/308:35:043
2331599172,10cyclictest0-21swapper/308:20:133
2331599171,13cyclictest0-21swapper/308:05:283
2331599171,13cyclictest0-21swapper/307:10:223
2331599171,10cyclictest0-21swapper/307:50:223
23315991710,4cyclictest33-21ksoftirqd/309:05:183
2328799178,7cyclictest12-21ksoftirqd/007:15:140
2328799172,10cyclictest16187-21awk08:10:010
2328799171,6cyclictest2944-21ssh10:00:170
2328799171,3cyclictest131rcu_preempt07:40:020
23287991711,4cyclictest0-21swapper/007:10:270
2328799171,12cyclictest17095-21df09:50:180
2328799170,9cyclictest7552-21kthreadcore07:25:200
2328799167,7cyclictest24966-21perl07:45:170
2328799162,11cyclictest20551-21irqrtprio07:40:170
2328799162,10cyclictest0-21swapper/008:35:040
2328799162,10cyclictest0-21swapper/008:35:040
2328799161,12cyclictest32550-21kthreadcore08:25:200
2328799161,12cyclictest32550-21kthreadcore08:25:190
2331599152,7cyclictest0-21swapper/307:40:123
2331599151,12cyclictest0-21swapper/308:10:243
2331599131,7cyclictest0-21swapper/307:15:193
2331599121,9cyclictest0-21swapper/307:55:193
2331599101,6cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional