You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 21:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Jul 06, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
582799652,34cyclictest6553-21ssh12:35:141
583699551,35cyclictest14093-21ssh09:40:122
5836994732,9cyclictest4361-21apt-get11:40:132
5846994624,14cyclictest35966irq/58-eth010:50:153
5846994411,21cyclictest0-21swapper/309:30:143
5846994411,21cyclictest0-21swapper/309:30:133
582799442,31cyclictest20254-21ssh10:55:141
5846994331,8cyclictest35966irq/58-eth010:10:143
584699432,5cyclictest0-21swapper/309:15:133
584699421,2cyclictest0-21swapper/309:10:143
5819994035,3cyclictest0-21swapper/009:25:110
5819994035,3cyclictest0-21swapper/009:25:110
584699395,23cyclictest35966irq/58-eth012:00:153
584699392,2cyclictest0-21swapper/312:35:143
583699396,21cyclictest0-21swapper/208:55:132
583699391,27cyclictest13396-21ssh11:45:122
584699382,24cyclictest35966irq/58-eth008:10:133
584699381,25cyclictest35966irq/58-eth007:15:133
584699374,21cyclictest35966irq/58-eth011:15:093
583699373,23cyclictest8821-21ssh09:55:122
583699371,24cyclictest28239-21ssh09:30:142
583699371,24cyclictest28239-21ssh09:30:132
584699362,23cyclictest35966irq/58-eth012:30:133
5836993613,18cyclictest0-21swapper/211:00:142
5836993613,18cyclictest0-21swapper/211:00:132
5827993618,10cyclictest5818-21cyclictest11:40:131
584699352,16cyclictest35966irq/58-eth008:30:123
583699359,10cyclictest9686-21ssh11:25:142
583699352,20cyclictest27346irq/40-dwc2_hso07:30:122
581999359,4cyclictest12-21ksoftirqd/011:45:130
584699342,23cyclictest35966irq/58-eth009:40:113
584699341,7cyclictest0-21swapper/310:00:123
584699341,4cyclictest0-21swapper/311:05:133
584699341,4cyclictest0-21swapper/311:05:133
5836993427,3cyclictest30153-21apt-get07:35:132
5836993418,3cyclictest3746-21apt-get08:15:142
5836993411,17cyclictest0-21swapper/209:45:122
5836993313,12cyclictest0-21swapper/208:05:122
5827993311,16cyclictest5830-21diskmemload11:05:131
5827993311,16cyclictest5830-21diskmemload11:05:121
584699321,6cyclictest0-21swapper/310:40:133
583699321,14cyclictest14082-21grep09:20:302
584699311,3cyclictest0-21swapper/312:15:143
583699310,14cyclictest0-21swapper/212:35:222
581999314,24cyclictest3463-21ssh10:10:120
584699303,6cyclictest0-21swapper/307:10:153
584699302,3cyclictest0-21swapper/307:30:113
584699301,5cyclictest0-21swapper/312:10:143
583699308,14cyclictest0-21swapper/210:36:032
583699306,21cyclictest4862-21/usr/sbin/munin09:35:152
583699303,11cyclictest7809-21cut07:10:212
583699302,23cyclictest27346irq/40-dwc2_hso10:45:122
582799302,22cyclictest7486-21ssh10:30:141
582799302,20cyclictest14864-21ssh11:25:541
581999306,18cyclictest11234-21ssh10:50:140
581999303,24cyclictest5820-21rm11:05:130
581999303,24cyclictest5820-21rm11:05:120
584699293,20cyclictest35966irq/58-eth009:50:123
584699291,3cyclictest0-21swapper/308:20:143
583699299,8cyclictest30843-21sh10:25:152
583699294,18cyclictest27346irq/40-dwc2_hso08:30:122
583699293,18cyclictest27346irq/40-dwc2_hso09:50:122
583699293,18cyclictest23987-21diskmemload11:56:262
583699293,16cyclictest27346irq/40-dwc2_hso08:25:122
583699293,12cyclictest27346irq/40-dwc2_hso10:00:122
583699292,20cyclictest0-21swapper/207:25:132
583699292,19cyclictest18707-21ssh11:30:122
5836992916,9cyclictest7455-21apt-get10:30:132
5836992915,10cyclictest550-21apt-get11:20:142
5836992914,10cyclictest27722-21ssh11:35:102
582799292,18cyclictest4910-21diskmemload09:35:121
581999293,17cyclictest4899-21ssh09:35:120
584699282,6cyclictest0-21swapper/311:35:103
5846992816,9cyclictest35966irq/58-eth012:05:143
584699281,5cyclictest0-21swapper/308:40:283
583699289,14cyclictest22450-21apt-get11:50:112
583699289,14cyclictest22450-21apt-get11:50:102
583699287,13cyclictest0-21swapper/211:13:392
583699282,7cyclictest0-21swapper/209:25:132
583699282,7cyclictest0-21swapper/209:25:122
582799282,19cyclictest20493-21grep09:25:171
582799282,19cyclictest20493-21grep09:25:171
581999282,21cyclictest25081-21ssh11:32:170
581999282,20cyclictest14659-21ps10:15:180
581999282,14cyclictest151rcuc/009:15:120
581999281,18cyclictest6546-21sh12:35:120
584699273,16cyclictest0-21swapper/310:30:193
584699272,6cyclictest0-21swapper/307:40:133
584699272,6cyclictest0-21swapper/307:40:133
584699271,8cyclictest0-21swapper/311:00:133
584699271,8cyclictest0-21swapper/311:00:133
584699271,4cyclictest0-21swapper/309:55:273
583699276,12cyclictest0-21swapper/210:44:492
583699274,17cyclictest0-21swapper/209:10:152
5836992719,3cyclictest5804-21apt-get11:05:132
5836992719,3cyclictest5804-21apt-get11:05:132
582799275,18cyclictest21-21ksoftirqd/111:50:111
582799275,18cyclictest21-21ksoftirqd/111:50:101
584699268,10cyclictest35966irq/58-eth012:25:143
584699263,16cyclictest0-21swapper/311:45:183
584699261,5cyclictest0-21swapper/310:59:353
583699265,8cyclictest0-21swapper/210:25:012
583699263,17cyclictest27346irq/40-dwc2_hso07:15:142
583699263,15cyclictest27346irq/40-dwc2_hso10:50:132
5836992620,3cyclictest3453-21apt-get10:10:122
5836992620,3cyclictest20232-21apt-get10:55:122
583699261,16cyclictest27346irq/40-dwc2_hso10:15:122
582799268,12cyclictest21-21ksoftirqd/109:15:121
584699254,14cyclictest35966irq/58-eth009:45:123
584699251,8cyclictest0-21swapper/311:10:123
584699251,6cyclictest0-21swapper/309:00:223
584699251,5cyclictest0-21swapper/311:51:053
584699251,5cyclictest0-21swapper/311:51:053
584699251,5cyclictest0-21swapper/309:25:153
584699251,5cyclictest0-21swapper/309:25:143
584699251,4cyclictest0-21swapper/311:40:273
584699251,3cyclictest0-21swapper/311:30:213
584699251,3cyclictest0-21swapper/311:25:283
584699251,3cyclictest0-21swapper/311:20:203
583699254,9cyclictest0-21swapper/212:30:552
5836992520,2cyclictest17127-21apt-get12:05:162
583699251,9cyclictest27651-21ntp_states11:15:242
583699251,8cyclictest25035-21awk12:25:292
5836992510,6cyclictest3012-21ssh09:15:212
5836992510,13cyclictest16791-21latency_hist12:05:022
5836992510,11cyclictest27-21ksoftirqd/210:06:192
581999259,3cyclictest131rcu_preempt09:55:310
584699247,14cyclictest0-21swapper/311:55:073
584699241,6cyclictest0-21swapper/309:20:173
584699241,5cyclictest0-21swapper/310:48:063
584699241,5cyclictest0-21swapper/310:29:273
584699241,5cyclictest0-21swapper/309:35:293
584699241,5cyclictest0-21swapper/309:05:193
584699241,5cyclictest0-21swapper/308:55:233
584699241,5cyclictest0-21swapper/308:15:183
584699241,5cyclictest0-21swapper/307:35:163
584699241,4cyclictest0-21swapper/310:23:193
584699241,4cyclictest0-21swapper/310:05:283
584699241,3cyclictest0-21swapper/310:35:203
584699241,3cyclictest0-21swapper/310:15:183
584699241,3cyclictest0-21swapper/308:05:293
584699241,3cyclictest0-21swapper/307:45:223
583699243,8cyclictest0-21swapper/212:21:512
583699243,15cyclictest27346irq/40-dwc2_hso07:20:112
583699243,13cyclictest27346irq/40-dwc2_hso08:00:112
583699242,16cyclictest27346irq/40-dwc2_hso08:20:132
583699242,15cyclictest27346irq/40-dwc2_hso12:10:122
5836992411,10cyclictest21525-21cron08:00:012
581999244,17cyclictest26924-21ssh10:05:120
581999242,15cyclictest25928-21kworker/u8:111:10:130
5819992414,5cyclictest12-21ksoftirqd/008:00:130
584699231,8cyclictest0-21swapper/312:20:213
584699231,5cyclictest0-21swapper/308:50:163
584699231,3cyclictest0-21swapper/308:40:013
584699231,3cyclictest0-21swapper/307:20:143
583699239,12cyclictest23397-21cat09:05:292
583699235,9cyclictest31095-21load08:40:232
583699233,13cyclictest27346irq/40-dwc2_hso08:10:132
583699232,17cyclictest27346irq/40-dwc2_hso12:15:152
583699232,14cyclictest27346irq/40-dwc2_hso08:35:122
582799231,3cyclictest131rcu_preempt10:10:121
581999236,3cyclictest131rcu_preempt10:00:210
581999232,18cyclictest6276-21rm12:35:030
581999232,17cyclictest18720-21kthreadcore07:20:210
581999231,4cyclictest18321-21munin-run11:30:020
584699221,5cyclictest0-21swapper/307:30:003
584699221,3cyclictest0-21swapper/308:25:133
583699225,13cyclictest8711-21wc08:50:292
582799226,3cyclictest131rcu_preempt12:26:071
582799223,17cyclictest20460-21kthreadcore08:30:201
582799223,16cyclictest2250-21cpuspeed10:45:141
582799222,5cyclictest53646usb-storage09:40:061
582799222,17cyclictest3971-21kthreadcore07:40:201
582799222,17cyclictest3971-21kthreadcore07:40:191
582799222,17cyclictest27505-21kthreadcore10:40:201
582799222,17cyclictest2481-21latency_hist12:15:011
582799222,17cyclictest20657-21kthreadcore09:05:201
582799222,17cyclictest13284-21kthreadcore07:15:231
582799222,17cyclictest10535-21kthreadcore08:55:191
582799222,15cyclictest23987-21diskmemload10:18:171
581999222,18cyclictest20915-21kthreadcore09:05:200
581999222,17cyclictest19961-21kthreadcore07:55:220
581999221,6cyclictest31898-21if_eth111:55:190
5819992211,3cyclictest12-21ksoftirqd/010:40:180
584699211,5cyclictest0-21swapper/307:55:213
584699211,5cyclictest0-21swapper/307:50:153
584699211,3cyclictest0-21swapper/308:45:223
584699211,3cyclictest0-21swapper/308:00:183
583699219,9cyclictest0-21swapper/207:45:132
583699213,9cyclictest1693-21kthreadcore08:45:212
583699213,15cyclictest2691-21cut07:40:152
583699213,15cyclictest2691-21cut07:40:152
583699213,10cyclictest14587-21find09:00:122
582799219,7cyclictest0-21swapper/111:30:551
582799217,3cyclictest131rcu_preempt11:20:021
582799213,16cyclictest17356-21fschecks_time12:05:181
582799213,15cyclictest25784-21kthreadcore08:35:221
582799212,17cyclictest16686-21kthreadcore07:20:171
582799212,17cyclictest15720-21kthreadcore11:10:191
582799212,16cyclictest4088-21kthreadcore12:15:201
582799212,16cyclictest30594-21irqcore07:35:191
582799212,15cyclictest13708-21kthreadcore07:50:201
582799211,17cyclictest23987-21diskmemload11:35:241
581999217,3cyclictest131rcu_preempt10:20:160
581999213,3cyclictest23987-21diskmemload10:25:200
581999213,15cyclictest6994-21cat12:15:300
581999213,15cyclictest32577-21irqcore09:50:170
581999213,15cyclictest23356-21irqcore08:35:180
581999212,17cyclictest22535-21kthreadcore10:55:210
581999212,16cyclictest8383-21latency_hist08:20:010
581999212,16cyclictest6471-21irqcore07:10:190
581999212,16cyclictest3598-21kthreadcore07:40:190
581999212,16cyclictest3598-21kthreadcore07:40:180
581999212,16cyclictest32166-21kthreadcore08:10:180
581999212,16cyclictest24123-21fschecks_count11:15:130
581999212,15cyclictest569-21ssh11:20:130
581999211,6cyclictest26220-21df12:10:150
581999211,17cyclictest7166-21kthreadcore08:50:210
581999211,17cyclictest30586-21kthreadcore11:00:210
581999211,17cyclictest30586-21kthreadcore11:00:200
581999211,16cyclictest17042-21irqcore10:35:170
583699202,9cyclictest5959-21latency_hist07:10:022
583699201,10cyclictest12257-21tune2fs07:50:162
582799209,3cyclictest21-21ksoftirqd/109:51:421
582799207,7cyclictest0-21swapper/110:05:121
582799204,14cyclictest31868-21kthreadcore12:30:211
582799204,13cyclictest11390-21apt-get12:20:021
582799203,14cyclictest30310-21kthreadcore09:30:201
582799203,14cyclictest30310-21kthreadcore09:30:201
582799202,16cyclictest6009-21kthreadcore08:50:191
582799202,16cyclictest28332-21kthreadcore07:30:201
582799202,16cyclictest26079-21kthreadcore09:45:211
582799202,16cyclictest18379-21wc09:00:281
582799202,15cyclictest28774-21kthreadcore08:05:201
5827992013,4cyclictest0-21swapper/111:55:191
5827992012,6cyclictest26020-21cut09:10:211
5827992012,4cyclictest0-21swapper/109:20:141
582799201,17cyclictest21415-21ls10:00:231
582799201,12cyclictest786-21cut11:20:161
582799200,17cyclictest25478-21ls08:00:261
581999203,14cyclictest28705-21irqcore09:30:180
581999203,14cyclictest28705-21irqcore09:30:180
581999203,14cyclictest25737-21irqcore07:30:170
581999203,14cyclictest16074-21ssh09:40:200
581999203,14cyclictest12498-21grep08:20:300
581999202,16cyclictest26191-21kthreadcore09:45:220
581999202,16cyclictest15231-21kthreadcore07:50:220
581999202,16cyclictest14894-21kthreadcore08:25:200
581999202,16cyclictest10075-21kthreadcore07:45:210
581999202,15cyclictest18145-21processes09:00:260
581999201,16cyclictest27264-21cut09:10:230
581999201,16cyclictest21704-21kthreadcore07:25:190
581999201,16cyclictest21018-21kthreadcore08:30:220
5819992011,6cyclictest18173-21sshd09:24:170
581999201,16cyclictest14703-21kthreadcore12:20:230
581999201,15cyclictest11056-21hwlatdetect07:15:170
584699191,5cyclictest0-21swapper/307:10:023
582799193,13cyclictest28298-21ps08:40:181
582799192,15cyclictest9263-21kthreadcore07:45:201
582799192,15cyclictest11529-21kthreadcore09:55:211
582799192,14cyclictest882-21kthreadcore08:10:201
582799192,14cyclictest10179-21kthreadcore08:20:201
582799192,13cyclictest5928-21latency_hist07:10:031
582799191,9cyclictest0-21swapper/110:38:071
582799190,16cyclictest29367-21ls11:00:151
582799190,16cyclictest29367-21ls11:00:141
581999194,11cyclictest12-21ksoftirqd/010:30:140
581999193,13cyclictest8780-21sed12:00:200
581999193,13cyclictest29954-21kthreadcore08:40:200
581999192,15cyclictest9876-21cut08:55:160
581999191,16cyclictest28432-21kthreadcore08:05:200
581999191,15cyclictest25732-21users12:25:310
581999191,13cyclictest23987-21diskmemload11:50:120
581999191,13cyclictest23987-21diskmemload11:50:120
581999191,13cyclictest17140-21ssh12:05:140
582799183,12cyclictest1170-21cut08:45:201
582799182,14cyclictest7536-21grep08:15:301
582799182,13cyclictest9551-21tr12:00:231
582799182,13cyclictest20780-21grep07:55:291
582799182,13cyclictest13879-21cut08:25:181
582799182,13cyclictest10608-21cron07:15:011
582799182,11cyclictest22122-21ssh10:20:181
5827991811,4cyclictest30808-21/usr/sbin/munin10:25:171
581999182,14cyclictest5938-21ntp_states10:45:270
581999181,15cyclictest824-21kernelversion08:45:190
581999181,15cyclictest30795-21kernelversion07:35:190
581999181,13cyclictest4382-21ssh11:40:120
5819991810,5cyclictest31195-21load11:35:200
582799173,12cyclictest11446-21fschecks_time10:50:161
582799172,13cyclictest21379-21kthreadcore07:25:191
581999162,12cyclictest5830-21sort07:10:010
582799151,11cyclictest13453-21sort11:45:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional