You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Fri Apr 24, 2026 12:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
14911995816,31cyclictest131rcu_preempt12:15:120
14911995816,31cyclictest131rcu_preempt12:15:110
1492799564,4cyclictest27-21ksoftirqd/212:00:112
14935995528,15cyclictest35966irq/58-eth012:10:133
1492799541,8cyclictest0-21swapper/208:20:112
1493599534,30cyclictest35966irq/58-eth011:45:133
14935995217,15cyclictest35966irq/58-eth010:05:123
1491999501,44cyclictest21040-21apt-get07:50:111
1491999493,37cyclictest13770-21ssh09:35:121
1491999482,42cyclictest0-21swapper/108:30:121
1491999461,25cyclictest7032-21tune2fs11:55:161
1492799451,34cyclictest0-21swapper/210:30:102
1491999452,37cyclictest15787-21apt-get12:00:121
1491999441,34cyclictest24671-21apt-get10:35:141
1491999441,2cyclictest0-21swapper/108:35:121
1491199444,5cyclictest0-21swapper/012:20:130
1491199443,5cyclictest131rcu_preempt10:15:110
1491999431,38cyclictest0-21swapper/108:10:111
1491999431,37cyclictest17460-21apt-get11:25:111
1491199435,24cyclictest3029-21ssh10:05:110
1493599424,22cyclictest35966irq/58-eth010:50:123
1492799422,21cyclictest7017-21ssh11:55:172
1491999421,36cyclictest0-21swapper/110:55:111
1491999413,35cyclictest0-21swapper/108:00:121
1491999412,36cyclictest26432-21ssh10:00:121
1491999411,35cyclictest1610-21apt-get12:10:141
1491999411,2cyclictest0-21swapper/112:05:121
1491999411,27cyclictest13200-21tr11:40:201
1491999411,25cyclictest12807-21sh10:27:031
1492799402,23cyclictest21078-21ssh11:45:132
1491999402,34cyclictest17475-21apt-get08:20:121
1491199402,25cyclictest17345-21ssh09:55:120
1491999393,32cyclictest0-21swapper/112:35:131
1491999393,31cyclictest18738-21apt-get08:55:111
1491999392,34cyclictest10591-21apt-get12:15:121
1491999392,34cyclictest10591-21apt-get12:15:111
1491999391,3cyclictest0-21swapper/109:50:141
1491999391,3cyclictest0-21swapper/109:05:131
1491999391,36cyclictest29459-21apt-get07:25:121
1491999391,25cyclictest11407-21latency_hist07:40:031
1491199392,30cyclictest5513-21ssh12:30:140
1492799387,22cyclictest27-21ksoftirqd/208:55:122
1491999382,33cyclictest0-21swapper/109:45:121
1491999382,33cyclictest0-21swapper/109:45:111
1491999382,27cyclictest9956-21ssh09:15:121
1491999381,24cyclictest8103-21kthreadcore11:00:211
1491999381,22cyclictest6655-21ntp_192.168.11410:05:241
1491999380,3cyclictest22764-21apt-get11:10:121
1491999380,24cyclictest7722-21cat11:35:301
1492799372,3cyclictest131rcu_preempt09:40:112
1491999371,32cyclictest20997-21apt-get10:15:121
1491999371,31cyclictest19239-21ssh10:50:131
1491999371,25cyclictest32247-21unixbench_singl09:25:291
1491999361,2cyclictest0-21swapper/108:05:131
1491999361,2cyclictest0-21swapper/107:30:111
1491999361,25cyclictest481266ssh09:30:141
1491999361,24cyclictest22384-21sh10:32:451
1491999361,22cyclictest30020-21sh11:30:231
1491999361,19cyclictest11707-21ssh12:32:071
1491999360,2cyclictest23623-21apt-get09:00:131
14927993510,21cyclictest27-21ksoftirqd/207:50:112
1491999352,28cyclictest22782-21apt-get09:40:111
1491999351,4cyclictest0-21swapper/110:10:121
1491999351,30cyclictest9233-21kthreadcore11:20:191
1491999351,30cyclictest9233-21kthreadcore11:20:181
1491999351,30cyclictest22621-21if_err_eth008:25:171
1491999351,30cyclictest22621-21if_err_eth008:25:161
1491999351,29cyclictest0-21swapper/107:15:131
1491199351,30cyclictest15807-21ssh12:00:110
1491999341,3cyclictest0-21swapper/107:55:141
1491999341,30cyclictest27729-21latency_hist09:25:021
1491999341,2cyclictest0-21swapper/108:45:121
1491999341,29cyclictest17493-21mailstats08:50:251
1491999341,29cyclictest13771-21apt-get11:05:141
1491999340,4cyclictest17326-21apt-get09:55:101
1492799331,25cyclictest4701-21ssh09:30:112
1491999339,22cyclictest2062-21ssh12:26:331
1491999331,29cyclictest17018-21kthreadcore07:45:191
1491999331,28cyclictest30325-21fschecks_time10:20:161
1491999331,28cyclictest30325-21fschecks_time10:20:151
1491999331,28cyclictest23324-21sshd12:20:231
1491999331,28cyclictest12296-21tr07:40:191
1491999331,28cyclictest10527-21fschecks_time10:45:151
1491999331,21cyclictest0-21swapper/111:52:171
1491199332,23cyclictest17485-21ssh11:25:140
14935993216,10cyclictest35966irq/58-eth009:40:133
1491999321,28cyclictest8060-21timerandwakeup08:40:281
1491999321,28cyclictest2968-21awk11:15:241
1491999321,28cyclictest29217-21sshd07:24:171
1491999321,28cyclictest17189-21users08:15:311
1491999321,20cyclictest7414-21sh10:42:041
1491199329,6cyclictest0-21swapper/008:20:110
1493599313,17cyclictest35966irq/58-eth011:55:123
1493599313,13cyclictest0-21swapper/312:20:133
1492799312,18cyclictest27346irq/40-dwc2_hso07:40:152
1492799311,24cyclictest0-21swapper/209:55:122
1492799311,12cyclictest21003-21ssh10:15:102
1491999310,27cyclictest921-21ssh09:10:121
1491199311,22cyclictest4900-21cut12:10:220
1492799305,16cyclictest27346irq/40-dwc2_hso11:40:122
1491999301,26cyclictest15712-21irqstats07:10:161
1491199309,12cyclictest14899-21cyclictest07:15:130
1491199305,22cyclictest12-21ksoftirqd/009:15:130
1491199302,21cyclictest22776-21ssh11:10:130
1492799292,11cyclictest0-21swapper/212:30:142
14927992918,9cyclictest0-21swapper/209:25:142
1491999291,25cyclictest24832-21/usr/sbin/munin11:45:271
14911992922,4cyclictest12-21ksoftirqd/010:55:120
1493599288,10cyclictest0-21swapper/311:50:123
1492799282,18cyclictest27346irq/40-dwc2_hso11:10:112
1491199287,3cyclictest131rcu_preempt09:35:130
1491199282,18cyclictest7013-21cut11:55:170
1493599273,10cyclictest0-21swapper/312:04:243
1493599272,13cyclictest0-21swapper/311:00:123
14935992716,7cyclictest0-21swapper/312:05:143
1493599271,17cyclictest35966irq/58-eth009:35:123
1493599271,16cyclictest0-21swapper/309:50:123
1491199274,18cyclictest12127-21rm11:40:130
1491199272,22cyclictest12-21ksoftirqd/008:45:140
1491199272,19cyclictest32400-21ssh12:25:240
1491199271,17cyclictest26470-21ssh11:30:120
1493599266,14cyclictest0-21swapper/309:25:133
1493599263,15cyclictest0-21swapper/309:45:113
1493599263,15cyclictest0-21swapper/309:45:113
1493599263,14cyclictest35966irq/58-eth011:35:123
14927992611,9cyclictest27-21ksoftirqd/212:05:172
1492799261,16cyclictest27346irq/40-dwc2_hso12:25:112
1491999261,21cyclictest14899-21cyclictest07:10:021
1491199262,13cyclictest0-21swapper/009:40:120
14911992616,7cyclictest536-21usb-storage09:00:110
1493599259,11cyclictest35966irq/58-eth010:55:133
1492799257,3cyclictest131rcu_preempt10:20:132
1492799257,3cyclictest131rcu_preempt10:20:132
1492799254,14cyclictest27-21ksoftirqd/208:25:142
1492799254,14cyclictest27-21ksoftirqd/208:25:132
1492799252,15cyclictest506-21diskmemload10:10:132
14927992512,10cyclictest11196-21ssh11:02:222
14927992510,6cyclictest27-21ksoftirqd/212:15:192
14927992510,6cyclictest27-21ksoftirqd/212:15:192
1491199251,10cyclictest1270-21ssh10:40:130
1493599242,10cyclictest0-21swapper/312:25:243
14935992415,6cyclictest35966irq/58-eth011:40:243
1492799243,16cyclictest27346irq/40-dwc2_hso09:10:112
1492799242,8cyclictest27002-21ssh12:23:242
1492799242,8cyclictest0-21swapper/209:20:122
1492799242,10cyclictest24697-21ssh10:35:142
14927992412,9cyclictest1508-21ssh10:03:382
14927992412,10cyclictest27-21ksoftirqd/210:50:122
14927992410,11cyclictest27-21ksoftirqd/212:35:192
1491199243,16cyclictest14614-21ssh12:35:120
1491199242,18cyclictest24929-21ssh12:05:130
1493599232,15cyclictest35966irq/58-eth008:40:123
1492799239,10cyclictest27-21ksoftirqd/210:45:132
1492799234,8cyclictest31763-21ssh09:45:122
1492799234,8cyclictest31763-21ssh09:45:112
1492799234,15cyclictest27346irq/40-dwc2_hso12:10:132
1492799233,15cyclictest27346irq/40-dwc2_hso09:15:152
14927992316,5cyclictest0-21swapper/211:20:132
14927992316,5cyclictest0-21swapper/211:20:122
14927992313,8cyclictest0-21swapper/211:35:322
14927992312,9cyclictest6876-21ssh11:18:412
14927992311,9cyclictest27-21ksoftirqd/209:36:412
14927992311,10cyclictest0-21swapper/210:42:452
1491199239,3cyclictest12-21ksoftirqd/008:50:090
1491199231,3cyclictest131rcu_preempt11:20:290
1491199231,3cyclictest131rcu_preempt11:20:280
1491199230,17cyclictest31780-21ssh11:15:130
1493599224,14cyclictest0-21swapper/311:05:133
14935992214,5cyclictest35966irq/58-eth011:25:133
1493599221,19cyclictest0-21swapper/312:15:123
1493599221,19cyclictest0-21swapper/312:15:113
1492799229,9cyclictest0-21swapper/208:35:122
1492799229,10cyclictest9721-21tr10:25:212
1492799226,13cyclictest28236-21ssh10:55:132
1492799225,13cyclictest27346irq/40-dwc2_hso07:25:112
1492799223,12cyclictest0-21swapper/208:40:132
1492799222,6cyclictest0-21swapper/210:05:132
14927992212,7cyclictest27-21ksoftirqd/207:55:192
14927992211,9cyclictest27-21ksoftirqd/209:00:242
1492799221,17cyclictest17703-21ssh11:25:152
1491199224,8cyclictest131rcu_preempt07:10:190
1491199221,8cyclictest3984-21kthreadcore07:30:200
14911992216,3cyclictest0-21swapper/011:05:130
1491199221,5cyclictest53646usb-storage10:45:120
1491199221,4cyclictest131rcu_preempt07:45:270
1491199220,2cyclictest131rcu_preempt08:00:170
1493599214,12cyclictest0-21swapper/310:25:133
1493599212,15cyclictest0-21swapper/309:55:133
14935992113,6cyclictest35966irq/58-eth011:30:113
14935992112,6cyclictest35966irq/58-eth012:30:123
1493599211,12cyclictest35966irq/58-eth010:20:143
1493599211,12cyclictest35966irq/58-eth010:20:133
1492799218,9cyclictest27-21ksoftirqd/207:10:172
1492799217,5cyclictest6854-21grep08:05:292
1492799217,11cyclictest9633-21sh09:50:192
1492799214,13cyclictest27346irq/40-dwc2_hso08:00:132
1492799212,10cyclictest30378-21fschecks_time11:50:162
1492799211,5cyclictest11541-21timerwakeupswit08:10:282
14927992112,7cyclictest0-21swapper/209:05:302
1492799211,12cyclictest6400-21sh07:35:012
1492799211,11cyclictest15443-21kthreadcore08:15:212
1492799211,10cyclictest26668-21fschecks_time11:30:152
14927992110,3cyclictest27-21ksoftirqd/208:30:272
1491199215,11cyclictest14899-21cyclictest09:30:120
1491199213,2cyclictest131rcu_preempt09:55:010
1491199211,4cyclictest131rcu_preempt10:25:290
1491199211,4cyclictest131rcu_preempt10:00:190
1491199211,3cyclictest131rcu_preempt10:54:450
1491199211,3cyclictest131rcu_preempt09:20:310
1491199211,17cyclictest3340-21timerwakeupswit09:45:290
1491199211,17cyclictest3340-21timerwakeupswit09:45:280
1491199211,15cyclictest30096-21ssh10:20:140
1491199211,15cyclictest30096-21ssh10:20:130
1493599203,9cyclictest0-21swapper/310:10:123
14935992011,6cyclictest0-21swapper/311:10:173
1493599201,13cyclictest0-21swapper/311:15:123
14935992010,7cyclictest0-21swapper/310:35:283
1492799206,7cyclictest20168-21tail07:15:182
1492799204,13cyclictest14231-21irqcore11:05:182
1492799203,14cyclictest17754-21grep08:50:252
14927992010,2cyclictest27-21ksoftirqd/208:45:222
1491199203,12cyclictest0-21swapper/008:05:120
1491199202,10cyclictest131rcu_preempt11:53:000
1491199201,4cyclictest131rcu_preempt09:10:010
14911992014,4cyclictest0-21swapper/011:35:240
1491199201,3cyclictest131rcu_preempt09:13:590
1491199201,13cyclictest11122-21ls08:10:220
1493599199,8cyclictest0-21swapper/310:46:133
1493599191,12cyclictest0-21swapper/309:05:113
14935991910,6cyclictest0-21swapper/310:01:203
1492799192,8cyclictest7988-21kthreadcore07:35:192
1492799191,7cyclictest0-21swapper/207:20:142
1492799191,10cyclictest19146-21kthreadcore07:45:202
1491199191,8cyclictest131rcu_preempt10:10:310
1491199191,6cyclictest131rcu_preempt08:15:130
1491199191,4cyclictest131rcu_preempt11:01:020
14911991913,4cyclictest24648-21/usr/sbin/munin10:35:160
14911991913,4cyclictest0-21swapper/010:30:290
1491199191,2cyclictest131rcu_preempt08:40:140
1491199191,2cyclictest131rcu_preempt08:25:170
1491199191,2cyclictest131rcu_preempt08:25:160
1491199191,14cyclictest24625-21apt-get07:20:130
1491199190,2cyclictest131rcu_preempt11:45:330
1493599189,6cyclictest35966irq/58-eth009:30:123
1493599189,6cyclictest0-21swapper/309:22:443
1493599183,10cyclictest35966irq/58-eth009:15:123
1493599182,9cyclictest35966irq/58-eth012:35:153
1493599182,10cyclictest0-21swapper/308:50:113
1493599182,10cyclictest0-21swapper/308:30:113
1493599182,10cyclictest0-21swapper/308:15:143
14935991810,6cyclictest0-21swapper/310:32:533
14935991810,6cyclictest0-21swapper/309:12:503
14935991810,5cyclictest35966irq/58-eth010:15:113
14935991810,4cyclictest0-21swapper/308:25:133
14935991810,4cyclictest0-21swapper/308:25:123
1491199181,4cyclictest131rcu_preempt09:25:170
1491199181,2cyclictest131rcu_preempt08:30:170
1491199180,11cyclictest21270-21ls07:50:160
1491199180,11cyclictest14674-21ls07:40:230
1493599179,6cyclictest0-21swapper/310:42:003
1493599172,9cyclictest0-21swapper/308:00:133
1493599172,10cyclictest0-21swapper/307:30:133
1493599171,13cyclictest0-21swapper/311:20:233
1493599171,13cyclictest0-21swapper/311:20:223
1492799171,7cyclictest14954-21latency_hist07:10:012
1491199171,8cyclictest1258-21kthreadcore08:35:190
1491199171,5cyclictest25847-21cpu07:55:130
1491199171,3cyclictest131rcu_preempt07:40:010
1491199171,10cyclictest29567-21cut07:25:140
1493599162,9cyclictest0-21swapper/308:20:113
1493599161,9cyclictest0-21swapper/308:55:203
1493599161,9cyclictest0-21swapper/307:10:253
1493599161,12cyclictest0-21swapper/307:50:013
1493599161,10cyclictest0-21swapper/309:05:023
1491199160,13cyclictest21740-21cut08:55:210
1491199160,10cyclictest14951-21ls07:10:010
1493599152,9cyclictest0-21swapper/307:25:133
1493599151,9cyclictest35966irq/58-eth008:05:153
1493599151,10cyclictest0-21swapper/308:40:023
1493599145,6cyclictest0-21swapper/307:20:193
1493599143,7cyclictest0-21swapper/307:50:103
1493599141,7cyclictest0-21swapper/308:00:023
1493599141,7cyclictest0-21swapper/307:15:293
1493599141,10cyclictest0-21swapper/308:10:203
1493599132,4cyclictest0-21swapper/307:40:143
1493599131,7cyclictest0-21swapper/307:35:183
1493599123,7cyclictest0-21swapper/308:45:293
1493599111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional