You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 17:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Aug 20, 2026 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26964998070,7cyclictest35966irq/58-eth012:10:123
2694699643,50cyclictest26062-21ssh11:05:141
26964996253,6cyclictest35966irq/58-eth009:35:143
26964995642,11cyclictest35966irq/58-eth009:40:133
26938995114,3cyclictest131rcu_preempt12:25:130
2693899508,7cyclictest131rcu_preempt11:50:140
2696499492,29cyclictest35966irq/58-eth008:05:153
2693899479,7cyclictest131rcu_preempt11:40:140
2696499464,24cyclictest35966irq/58-eth011:30:123
26964994317,21cyclictest35966irq/58-eth009:25:133
26938994211,4cyclictest131rcu_preempt11:20:110
2696499412,28cyclictest35966irq/58-eth012:05:143
2693899407,8cyclictest12-21ksoftirqd/011:10:130
2696499394,21cyclictest35966irq/58-eth008:25:113
2695699384,9cyclictest27-21ksoftirqd/210:50:132
2694699382,26cyclictest13220-21ssh09:10:141
26946993817,17cyclictest21-21ksoftirqd/110:40:131
2693899384,10cyclictest12-21ksoftirqd/010:05:150
2696499368,17cyclictest35966irq/58-eth011:15:123
2696499362,20cyclictest35966irq/58-eth012:15:103
2696499355,20cyclictest35966irq/58-eth011:05:153
2696499353,22cyclictest35966irq/58-eth011:55:113
2696499352,19cyclictest35966irq/58-eth010:25:113
2694699357,19cyclictest21-21ksoftirqd/111:50:151
2693899340,26cyclictest26149-21ssh12:35:140
2696499333,5cyclictest0-21swapper/312:00:123
2696499333,5cyclictest0-21swapper/309:20:133
2696499333,19cyclictest35966irq/58-eth011:45:123
26964993311,18cyclictest0-21swapper/310:35:143
2694699333,24cyclictest29782-21ssh11:25:141
2696499321,3cyclictest0-21swapper/309:55:133
2696499313,3cyclictest0-21swapper/307:45:133
26964993113,14cyclictest35966irq/58-eth011:40:143
26956993110,8cyclictest131rcu_preempt11:10:122
2693899316,6cyclictest131rcu_preempt08:15:120
2693899311,6cyclictest12-21ksoftirqd/009:10:120
26938993111,13cyclictest12-21ksoftirqd/010:27:410
2696499303,3cyclictest0-21swapper/311:00:133
26956993016,10cyclictest261rcuc/212:20:132
2694699303,4cyclictest131rcu_preempt09:20:141
2694699301,17cyclictest29420-21kworker/u8:210:20:121
2693899308,5cyclictest131rcu_preempt10:50:120
2693899306,3cyclictest131rcu_preempt10:45:120
2696499295,12cyclictest35966irq/58-eth009:45:113
2696499291,3cyclictest0-21swapper/311:20:163
2696499291,14cyclictest35966irq/58-eth007:50:103
2695699298,11cyclictest0-21swapper/211:45:122
26956992924,3cyclictest27-21ksoftirqd/208:45:102
26956992924,3cyclictest27-21ksoftirqd/208:45:102
26956992911,10cyclictest27-21ksoftirqd/211:32:352
2694699290,23cyclictest12805-21diskmemload11:57:561
2693899296,3cyclictest131rcu_preempt09:15:110
2693899294,7cyclictest12-21ksoftirqd/009:55:130
2693899292,22cyclictest6322-21ssh11:30:120
26938992918,3cyclictest12805-21diskmemload09:30:210
2696499282,3cyclictest0-21swapper/311:25:143
26956992813,7cyclictest27-21ksoftirqd/212:08:032
2694699282,18cyclictest0-21swapper/109:30:121
2693899289,9cyclictest12-21ksoftirqd/010:35:140
2693899285,2cyclictest131rcu_preempt11:25:150
2693899283,2cyclictest131rcu_preempt08:00:130
26938992823,3cyclictest12805-21diskmemload11:45:120
2693899282,21cyclictest11546-21ssh09:45:110
26938992812,6cyclictest18493-21ssh11:35:220
2696499273,15cyclictest35966irq/58-eth009:05:123
2694699271,18cyclictest22335-21ssh12:15:101
2693899272,7cyclictest12-21ksoftirqd/007:25:120
26938992711,3cyclictest12-21ksoftirqd/012:10:300
26938992711,12cyclictest131rcu_preempt08:40:140
2696499268,12cyclictest0-21swapper/309:11:053
2696499263,14cyclictest35966irq/58-eth010:55:123
2696499262,5cyclictest0-21swapper/308:20:113
2696499261,6cyclictest0-21swapper/310:30:243
2696499261,6cyclictest0-21swapper/309:30:213
2696499261,4cyclictest0-21swapper/312:25:193
2696499261,15cyclictest35966irq/58-eth010:45:143
2695699268,12cyclictest27-21ksoftirqd/210:03:192
2695699267,6cyclictest131rcu_preempt11:20:002
2695699261,7cyclictest2419866ssh10:10:132
26956992611,6cyclictest27-21ksoftirqd/208:55:212
2694699269,10cyclictest21-21ksoftirqd/109:50:121
2694699263,6cyclictest131rcu_preempt12:09:561
26946992613,9cyclictest12805-21diskmemload10:15:131
2693899268,2cyclictest12-21ksoftirqd/010:31:460
2693899264,5cyclictest12-21ksoftirqd/011:00:130
2693899263,7cyclictest12-21ksoftirqd/011:15:140
2693899263,2cyclictest131rcu_preempt11:55:120
26938992619,4cyclictest131rcu_preempt12:30:130
26938992618,5cyclictest29562-21ssh09:35:260
2693899261,3cyclictest131rcu_preempt10:40:120
2696499256,4cyclictest0-21swapper/310:08:073
2696499254,3cyclictest0-21swapper/309:50:193
2696499254,3cyclictest0-21swapper/307:40:133
2696499252,20cyclictest35966irq/58-eth012:20:143
2696499251,5cyclictest0-21swapper/311:38:113
2696499251,5cyclictest0-21swapper/310:20:213
2696499251,3cyclictest0-21swapper/311:50:293
26964992512,10cyclictest35966irq/58-eth010:10:133
2695699259,7cyclictest27-21ksoftirqd/210:15:262
2695699256,16cyclictest261rcuc/211:00:232
2695699252,14cyclictest17040-21ssh12:30:132
2695699251,7cyclictest2138166ssh09:14:302
2695699251,7cyclictest1628666ssh11:52:262
26956992514,9cyclictest0-21swapper/209:25:132
26956992513,9cyclictest0-21swapper/212:35:292
26956992510,7cyclictest27-21ksoftirqd/210:43:222
2694699252,16cyclictest12805-21diskmemload10:25:111
2694699251,19cyclictest11804-21ssh11:15:141
2694699251,16cyclictest29609-21ssh09:55:141
2694699251,11cyclictest0-21swapper/109:29:201
2693899257,3cyclictest131rcu_preempt11:05:120
26938992511,8cyclictest12-21ksoftirqd/008:45:190
26938992511,8cyclictest12-21ksoftirqd/008:45:190
2696499241,7cyclictest0-21swapper/309:15:293
2696499241,7cyclictest0-21swapper/308:30:183
2696499241,5cyclictest0-21swapper/312:39:283
2696499241,5cyclictest0-21swapper/311:10:143
2696499241,5cyclictest0-21swapper/310:45:013
2696499241,5cyclictest0-21swapper/307:25:013
2696499241,4cyclictest0-21swapper/309:00:013
2696499241,4cyclictest0-21swapper/308:50:183
2696499241,4cyclictest0-21swapper/308:40:013
2696499241,4cyclictest0-21swapper/307:55:203
2695699249,7cyclictest27-21ksoftirqd/210:21:492
2695699249,7cyclictest27-21ksoftirqd/209:41:112
2695699249,7cyclictest27-21ksoftirqd/207:15:322
2695699247,6cyclictest131rcu_preempt07:35:122
2695699247,6cyclictest131rcu_preempt07:35:122
2695699247,3cyclictest131rcu_preempt08:10:172
2695699245,13cyclictest131rcu_preempt09:45:102
2695699242,19cyclictest27346irq/40-dwc2_hso08:50:162
26956992418,3cyclictest27-21ksoftirqd/211:20:122
2695699241,7cyclictest2811766ssh12:16:352
2695699241,5cyclictest2708166ssh09:15:332
26956992415,5cyclictest28845-21ssh12:00:192
26956992413,4cyclictest0-21swapper/209:55:132
26956992410,6cyclictest27-21ksoftirqd/211:38:362
26956992410,6cyclictest27-21ksoftirqd/211:08:592
2695699240,18cyclictest22392-21ssh10:45:132
2694699243,16cyclictest27952-21ssh10:30:151
2694699241,19cyclictest8042-21sh10:55:131
2694699241,18cyclictest6237-21ssh10:00:141
2694699240,21cyclictest27797-21ssh12:00:141
2694699240,14cyclictest12805-21diskmemload09:10:011
2693899249,7cyclictest12-21ksoftirqd/009:51:570
2693899249,2cyclictest12-21ksoftirqd/012:05:200
2693899248,2cyclictest12-21ksoftirqd/010:55:330
2693899246,5cyclictest131rcu_preempt08:10:120
2693899245,4cyclictest131rcu_preempt12:15:130
2693899244,4cyclictest131rcu_preempt08:05:150
26938992414,7cyclictest12-21ksoftirqd/007:40:130
26938992413,3cyclictest12-21ksoftirqd/008:20:170
26938992413,3cyclictest12-21ksoftirqd/007:50:190
2693899241,13cyclictest9813-21ntp_192.168.11410:00:240
2696499234,12cyclictest0-21swapper/310:50:133
2696499231,7cyclictest0-21swapper/309:05:023
2696499231,5cyclictest0-21swapper/312:30:203
2696499231,5cyclictest0-21swapper/310:00:073
2696499231,5cyclictest0-21swapper/308:45:143
2696499231,5cyclictest0-21swapper/308:45:143
2696499231,5cyclictest0-21swapper/308:10:283
2696499231,5cyclictest0-21swapper/307:30:203
2696499231,5cyclictest0-21swapper/307:25:163
2696499231,3cyclictest0-21swapper/310:15:213
2696499231,3cyclictest0-21swapper/307:20:023
2695699239,7cyclictest27-21ksoftirqd/210:28:142
2695699238,7cyclictest27-21ksoftirqd/207:55:142
2695699238,13cyclictest27-21ksoftirqd/209:23:062
2695699238,12cyclictest5851-21cat08:25:262
2695699236,13cyclictest27346irq/40-dwc2_hso10:35:152
2695699234,11cyclictest0-21swapper/208:15:122
2695699232,17cyclictest12805-21diskmemload12:25:132
2695699231,7cyclictest365766ssh11:27:162
2695699231,7cyclictest2407566ssh09:33:202
2695699231,7cyclictest2270766ssh11:55:262
2695699231,7cyclictest2122966ssh12:14:072
2695699231,6cyclictest287966ssh10:33:362
26956992312,4cyclictest0-21swapper/208:05:142
2695699231,10cyclictest27263-21kthreadcore11:40:222
26956992310,6cyclictest1904-21sshd08:24:562
26956992310,11cyclictest261rcuc/209:52:362
2694699238,6cyclictest131rcu_preempt12:37:421
2694699235,8cyclictest22555-21fschecks_count10:45:161
2694699232,15cyclictest5073-21ssh12:21:561
2694699231,10cyclictest0-21swapper/109:15:391
26946992310,8cyclictest21-21ksoftirqd/109:40:111
2693899238,8cyclictest0-21swapper/009:43:100
2693899236,3cyclictest131rcu_preempt10:15:120
2693899231,8cyclictest12985-21latency_hist09:10:010
26938992312,3cyclictest12-21ksoftirqd/012:05:010
26938992311,4cyclictest12-21ksoftirqd/007:35:220
26938992311,4cyclictest12-21ksoftirqd/007:35:220
2696499222,3cyclictest0-21swapper/308:15:173
2696499221,7cyclictest0-21swapper/307:15:013
2696499221,6cyclictest0-21swapper/308:40:153
2696499221,5cyclictest0-21swapper/307:35:143
2696499221,5cyclictest0-21swapper/307:35:143
2696499221,4cyclictest0-21swapper/308:00:293
2695699228,6cyclictest27-21ksoftirqd/210:57:212
2695699227,3cyclictest27-21ksoftirqd/208:00:142
2695699226,14cyclictest261rcuc/207:25:182
2695699225,14cyclictest26929-21cyclictest07:20:122
2695699221,7cyclictest93066snmpd07:12:442
2695699221,5cyclictest2640166kworker/u8:010:05:452
26956992211,8cyclictest0-21swapper/209:35:312
2695699221,16cyclictest12918-21ls09:10:012
2694699228,3cyclictest0-21swapper/109:48:441
2694699223,6cyclictest10193-21kworker/1:008:00:131
2694699223,3cyclictest0-21swapper/110:10:121
2694699221,5cyclictest0-21swapper/112:33:391
26946992214,5cyclictest0-21swapper/108:50:201
2693899228,3cyclictest12-21ksoftirqd/008:35:230
2693899226,7cyclictest12-21ksoftirqd/010:10:130
26938992215,4cyclictest0-21swapper/009:20:130
26938992212,3cyclictest12-21ksoftirqd/007:20:210
26938992210,9cyclictest12-21ksoftirqd/008:55:140
26938992210,4cyclictest12-21ksoftirqd/009:00:280
2695699211,7cyclictest913-21cpuspeed07:50:112
2695699211,6cyclictest0-21swapper/207:30:122
26956992116,3cyclictest0-21swapper/207:40:132
2695699211,15cyclictest27346irq/40-dwc2_hso08:30:132
2695699211,10cyclictest11846-21fschecks_count08:35:162
2694699216,4cyclictest4543-21df10:35:161
2694699216,3cyclictest131rcu_preempt07:20:021
2694699211,12cyclictest9003-21kthreadcore12:25:181
2693899219,4cyclictest12-21ksoftirqd/007:30:200
2693899219,2cyclictest131rcu_preempt08:25:110
2693899218,3cyclictest12-21ksoftirqd/007:55:170
2693899217,3cyclictest12-21ksoftirqd/010:20:120
2693899215,4cyclictest131rcu_preempt07:45:170
2693899214,3cyclictest131rcu_preempt12:20:190
2693899212,4cyclictest4112-21awk07:20:010
26938992110,3cyclictest12-21ksoftirqd/009:25:130
2695699203,15cyclictest3685-21if_eth009:00:162
2695699202,9cyclictest17389-21kthreadcore08:40:202
2695699201,8cyclictest30316-21kthreadcore07:45:192
2694699209,9cyclictest23796-21cron10:10:011
2694699207,5cyclictest0-21swapper/111:41:101
2694699206,5cyclictest131rcu_preempt11:45:291
2694699203,12cyclictest131rcu_preempt07:25:161
2694699202,12cyclictest131rcu_preempt08:10:191
2694699201,8cyclictest20658-21ssh11:00:251
2694699201,6cyclictest131rcu_preempt11:31:021
26946992012,2cyclictest0-21swapper/110:53:071
2694699201,10cyclictest7500-21kthreadcore07:55:201
26946992010,3cyclictest0-21swapper/109:36:331
26946992010,2cyclictest131rcu_preempt11:22:421
2694699200,4cyclictest131rcu_preempt08:55:121
2693899208,3cyclictest12-21ksoftirqd/008:30:130
2693899203,12cyclictest12-21ksoftirqd/007:10:110
2696499191,3cyclictest0-21swapper/307:10:023
2694699194,3cyclictest131rcu_preempt07:35:011
2694699193,11cyclictest131rcu_preempt07:50:211
2694699191,6cyclictest14950-21kthreadcore12:10:191
2694699191,3cyclictest131rcu_preempt11:35:041
26946991913,4cyclictest23421-21sendmail07:40:011
26946991913,4cyclictest23421-21sendmail07:40:011
26946991913,4cyclictest0-21swapper/111:10:131
2694699190,16cyclictest28210-21ls08:15:211
2693899193,12cyclictest12-21ksoftirqd/008:50:140
2695699181,13cyclictest27095-21grep07:10:022
2694699182,2cyclictest131rcu_preempt07:20:111
2694699181,9cyclictest7225-21if_eth008:30:181
26946991812,4cyclictest0-21swapper/107:10:141
2694699181,14cyclictest844-21munin-node08:05:131
2694699180,10cyclictest28791-21/usr/sbin/munin07:45:171
2693899187,2cyclictest12-21ksoftirqd/007:10:020
2694699174,6cyclictest131rcu_preempt08:40:311
2694699172,5cyclictest23750-21/usr/sbin/munin07:40:141
2694699171,8cyclictest24971-21switchtime08:45:251
2694699171,8cyclictest24971-21switchtime08:45:251
2694699171,7cyclictest12849-21kthreadcore08:35:201
26946991711,4cyclictest0-21swapper/108:25:141
2694699170,7cyclictest3588-21fschecks_time09:00:151
2694699165,3cyclictest131rcu_preempt08:20:191
2694699142,9cyclictest26978-21grep07:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional