You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-14 - 18:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Mar 14, 2026 12:45:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
471899593,12cyclictest131rcu_preempt12:10:121
471899593,12cyclictest131rcu_preempt12:10:121
471899583,42cyclictest21125-21ssh09:45:131
4734995737,12cyclictest28366irq/58-eth011:35:133
471899543,9cyclictest131rcu_preempt08:25:121
4706994812,5cyclictest24612-21apt-get10:05:110
472799443,18cyclictest24043-21ssh11:35:132
472799431,35cyclictest30816-21ssh12:15:112
472799411,31cyclictest11675-21ssh11:10:122
473499402,24cyclictest28366irq/58-eth012:15:113
471899395,5cyclictest131rcu_preempt07:10:121
470699393,31cyclictest10155-21rm10:15:120
471899387,10cyclictest131rcu_preempt12:05:121
470699384,31cyclictest3122-21ssh09:35:150
470699384,29cyclictest19159-21ssh10:20:120
470699382,32cyclictest21909-21ssh12:10:120
470699382,32cyclictest21909-21ssh12:10:120
473499371,19cyclictest28366irq/58-eth012:20:133
4734993612,12cyclictest28366irq/58-eth012:35:143
471899365,22cyclictest32020-21ssh09:15:131
471899365,22cyclictest32020-21ssh09:15:131
471899363,13cyclictest131rcu_preempt09:55:141
470699366,26cyclictest31807-21ssh10:45:130
473499351,20cyclictest28366irq/58-eth012:05:133
4727993412,9cyclictest131rcu_preempt10:35:152
471899343,4cyclictest131rcu_preempt08:05:141
470699341,25cyclictest17326-21ssh10:55:150
471899333,19cyclictest201rcuc/110:25:121
471899333,18cyclictest21-21ksoftirqd/108:10:121
4706993321,7cyclictest0-21swapper/008:05:140
4706993316,11cyclictest16454-21diskmemload12:25:150
472799322,13cyclictest28998-21grep09:30:202
471899321,4cyclictest131rcu_preempt08:40:121
471899321,4cyclictest131rcu_preempt08:40:121
470699327,6cyclictest12-21ksoftirqd/010:50:130
470699325,21cyclictest4012-21ssh12:00:150
470699323,17cyclictest1970-21ssh12:35:150
470699322,22cyclictest12073-21ssh09:40:130
470699322,22cyclictest12073-21ssh09:40:130
4706993218,6cyclictest0-21swapper/007:50:130
470699321,25cyclictest23092-21ssh09:10:110
473499314,22cyclictest0-21swapper/309:50:113
473499313,13cyclictest28366irq/58-eth009:15:123
473499313,13cyclictest28366irq/58-eth009:15:123
470699312,21cyclictest7420-21ssh12:20:140
472799307,18cyclictest27-21ksoftirqd/212:05:122
472799303,15cyclictest261rcuc/210:25:122
470699303,22cyclictest15537-21kworker/u8:210:25:130
470699302,21cyclictest2777-21diskmemload11:05:150
4706993021,5cyclictest0-21swapper/010:30:120
4706993019,5cyclictest0-21swapper/009:30:120
472799292,11cyclictest10251-21ssh09:38:202
472799292,10cyclictest20434-21cut11:50:202
470699299,6cyclictest12-21ksoftirqd/011:25:210
470699293,21cyclictest1166-21ssh10:10:120
470699293,21cyclictest1166-21ssh10:10:120
470699292,22cyclictest27379-21cut12:30:200
470699292,19cyclictest15142-21ssh11:30:130
470699291,23cyclictest22980-21kthreadcore11:15:200
472799283,18cyclictest2601-21kthreadcore10:10:182
472799283,18cyclictest2601-21kthreadcore10:10:182
4727992810,6cyclictest0-21swapper/210:31:312
471899284,3cyclictest131rcu_preempt11:35:121
471899282,19cyclictest22773-21ssh10:20:241
471899282,18cyclictest6662-21/usr/sbin/munin09:35:251
4718992818,7cyclictest21-21ksoftirqd/110:00:131
470699285,7cyclictest131rcu_preempt12:07:200
470699284,5cyclictest12-21ksoftirqd/009:05:290
470699282,19cyclictest5131-21ssh11:24:200
4706992810,8cyclictest12-21ksoftirqd/011:51:490
472799272,21cyclictest0-21swapper/209:28:202
472799272,20cyclictest5728-21ssh09:54:202
472799272,20cyclictest0-21swapper/210:54:202
472799272,19cyclictest0-21swapper/208:30:042
472799272,15cyclictest3376-21cut11:40:222
471899273,18cyclictest30159-21ssh09:50:131
471899273,17cyclictest20611-21ssh11:15:131
471899272,20cyclictest6115-21ssh11:25:141
4718992712,6cyclictest0-21swapper/107:20:121
470699277,4cyclictest131rcu_preempt11:10:270
470699274,5cyclictest0-21swapper/010:35:150
470699272,18cyclictest12-21ksoftirqd/011:00:140
470699272,18cyclictest12-21ksoftirqd/011:00:140
4706992716,4cyclictest0-21swapper/009:00:130
4706992713,7cyclictest0-21swapper/009:55:120
473499262,17cyclictest321rcuc/310:20:133
472799264,9cyclictest131rcu_preempt11:25:242
472799264,14cyclictest18328-21kthreadcore11:30:222
472799262,9cyclictest0-21swapper/208:35:142
472799262,8cyclictest14959-21ssh09:40:212
472799262,8cyclictest14959-21ssh09:40:202
472799262,21cyclictest26796-21kthreadcore08:05:212
472799262,19cyclictest10302-21ssh12:02:202
472799262,15cyclictest17719-21kthreadcore10:00:222
472799261,12cyclictest24622-21ssh10:05:122
4727992610,9cyclictest4696-21cyclictest08:00:112
4727992610,9cyclictest4696-21cyclictest08:00:112
471899261,5cyclictest21-21ksoftirqd/108:00:171
471899261,5cyclictest21-21ksoftirqd/108:00:161
4718992611,12cyclictest11286-21cron11:10:011
470699264,18cyclictest22791-21ssh10:40:120
4706992612,8cyclictest0-21swapper/009:25:120
4706992612,3cyclictest0-21swapper/012:17:540
4706992611,7cyclictest0-21swapper/009:45:130
4706992611,4cyclictest12-21ksoftirqd/011:55:220
473499253,10cyclictest0-21swapper/309:35:363
4734992512,11cyclictest33-21ksoftirqd/311:10:123
472799254,17cyclictest7322-21irqrtprio09:55:192
4727992512,6cyclictest27-21ksoftirqd/208:25:122
472799251,20cyclictest3709-21apt-get08:50:112
472799251,18cyclictest3516-21kthreadcore11:05:202
4718992512,4cyclictest21-21ksoftirqd/110:05:161
470699258,10cyclictest12-21ksoftirqd/007:10:140
470699254,6cyclictest12-21ksoftirqd/010:00:220
470699251,4cyclictest12-21ksoftirqd/008:15:130
470699251,18cyclictest0-21swapper/009:20:140
473499243,14cyclictest0-21swapper/311:50:203
4734992415,5cyclictest0-21swapper/311:25:123
472799244,9cyclictest48-21kcompactd011:20:042
472799242,19cyclictest31308-21cpu08:45:132
472799242,17cyclictest0-21swapper/212:38:192
4727992412,10cyclictest31830-21ssh12:32:262
472799241,17cyclictest0-21swapper/208:10:132
472799241,12cyclictest32019-21ssh09:15:132
472799241,12cyclictest32019-21ssh09:15:132
4727992411,11cyclictest4458-21ssh10:45:462
471899242,19cyclictest32419-21irqrtprio10:45:181
4718992414,6cyclictest22706-21diskmemload11:55:241
4718992410,3cyclictest131rcu_preempt12:15:291
470699249,3cyclictest131rcu_preempt07:35:150
470699249,3cyclictest131rcu_preempt07:35:150
470699246,4cyclictest131rcu_preempt08:30:210
470699245,3cyclictest131rcu_preempt08:00:140
470699245,3cyclictest131rcu_preempt08:00:140
473499233,12cyclictest0-21swapper/309:30:203
472799239,11cyclictest21084-21cut11:15:162
472799236,15cyclictest20141-21kthreadcore07:25:192
472799234,16cyclictest22161-21ssh12:10:162
472799234,16cyclictest22161-21ssh12:10:162
472799233,17cyclictest29025-21grep07:35:172
472799233,17cyclictest29025-21grep07:35:162
472799233,17cyclictest19731-21ssh10:20:182
472799232,6cyclictest0-21swapper/210:18:202
472799232,18cyclictest9621-21kthreadcore09:20:202
472799232,18cyclictest25075-21kthreadcore10:40:202
472799232,18cyclictest23237-21fschecks_count09:10:152
4727992310,11cyclictest25159-21ssh10:59:092
471899239,3cyclictest0-21swapper/107:30:151
471899238,3cyclictest131rcu_preempt09:30:021
471899237,2cyclictest131rcu_preempt07:55:311
471899236,4cyclictest131rcu_preempt12:30:221
471899235,5cyclictest29808-21sort11:20:161
471899232,18cyclictest24547-21kthreadcore10:40:191
471899231,7cyclictest8834-21unixbench_multi10:30:301
4718992311,3cyclictest21-21ksoftirqd/111:55:021
470699238,3cyclictest131rcu_preempt09:15:160
470699238,3cyclictest131rcu_preempt09:15:150
470699235,3cyclictest131rcu_preempt08:55:200
470699234,3cyclictest131rcu_preempt08:35:190
470699232,5cyclictest12-21ksoftirqd/011:35:220
470699232,4cyclictest12-21ksoftirqd/011:44:280
470699231,3cyclictest131rcu_preempt11:45:200
4706992313,7cyclictest0-21swapper/007:40:140
473499221,11cyclictest28366irq/58-eth010:25:123
472799228,11cyclictest7613-21cut12:20:172
472799224,8cyclictest131rcu_preempt07:50:202
472799223,8cyclictest0-21swapper/211:04:392
472799223,8cyclictest0-21swapper/211:04:392
472799223,17cyclictest29017-21kthreadcore11:55:192
472799223,16cyclictest25208-21latency_hist12:30:022
472799222,7cyclictest131rcu_preempt08:00:012
472799222,17cyclictest5807-21cron07:45:012
472799222,17cyclictest4272-21kthreadcore08:15:212
472799221,19cyclictest11334-21kthreadcore11:45:192
472799221,15cyclictest18079-21grep09:05:022
472799221,14cyclictest0-21swapper/207:30:122
472799221,11cyclictest131rcu_preempt08:25:012
471899229,4cyclictest131rcu_preempt10:55:121
471899228,2cyclictest131rcu_preempt07:35:211
471899228,2cyclictest131rcu_preempt07:35:211
471899226,5cyclictest16131-21ssh11:47:441
471899226,3cyclictest131rcu_preempt09:30:281
471899223,16cyclictest10128-21ssh10:15:101
471899221,9cyclictest131rcu_preempt09:10:251
471899221,7cyclictest1279-21diskstats10:10:141
471899221,7cyclictest1279-21diskstats10:10:141
4718992213,6cyclictest22389-21cron10:40:011
471899221,16cyclictest29720-21ls11:00:241
471899221,16cyclictest29720-21ls11:00:231
471899220,9cyclictest131rcu_preempt11:30:041
470699224,4cyclictest12-21ksoftirqd/007:45:190
470699224,3cyclictest131rcu_preempt07:30:270
470699222,5cyclictest12-21ksoftirqd/007:55:270
472799213,16cyclictest18058-21switchtime07:20:282
472799211,17cyclictest9145-21kernelversion08:55:182
472799211,17cyclictest21949-21tail09:05:292
472799211,16cyclictest8366-21grep07:10:262
472799211,10cyclictest6156-21cut07:45:152
472799211,10cyclictest22399-21kthreadcore09:45:192
472799211,10cyclictest13311-21cat07:15:292
471899219,3cyclictest21-21ksoftirqd/109:40:211
471899219,3cyclictest21-21ksoftirqd/109:40:201
471899212,9cyclictest131rcu_preempt07:45:271
471899212,5cyclictest131rcu_preempt12:35:141
471899212,16cyclictest9748-21tune2fs07:15:161
471899212,16cyclictest5050-21kthreadcore08:15:211
471899212,13cyclictest0-21swapper/111:10:121
4718992115,4cyclictest0-21swapper/110:50:231
4718992114,4cyclictest0-21swapper/108:35:181
471899211,3cyclictest131rcu_preempt09:00:121
4718992112,7cyclictest21-21ksoftirqd/108:55:191
471899211,18cyclictest2204-21kthreadcore07:40:201
471899211,17cyclictest672-21kthreadcore08:45:201
471899211,17cyclictest17063-21fschecks_time08:30:171
471899211,15cyclictest4433-21sh11:40:281
470699217,3cyclictest131rcu_preempt08:40:280
470699217,3cyclictest131rcu_preempt08:40:270
470699215,3cyclictest131rcu_preempt08:50:170
470699213,3cyclictest131rcu_preempt07:20:120
4706992110,5cyclictest12-21ksoftirqd/008:45:260
473499203,14cyclictest0-21swapper/309:25:213
473499201,14cyclictest28366irq/58-eth010:48:263
4734992011,4cyclictest0-21swapper/310:15:303
472799202,7cyclictest26498-21cpu08:40:142
472799202,7cyclictest26498-21cpu08:40:142
471899208,3cyclictest21-21ksoftirqd/112:25:231
471899207,3cyclictest22706-21diskmemload12:00:591
471899202,15cyclictest19873-21kthreadcore09:05:211
471899201,7cyclictest7545-21entropy12:20:161
471899201,16cyclictest7866-21kernelversion08:20:191
470699208,3cyclictest131rcu_preempt07:25:200
470699205,3cyclictest131rcu_preempt08:25:120
470699205,3cyclictest131rcu_preempt07:15:130
470699203,2cyclictest131rcu_preempt09:50:120
473499193,13cyclictest28366irq/58-eth012:25:323
473499193,11cyclictest0-21swapper/312:03:203
473499193,10cyclictest0-21swapper/309:40:213
473499193,10cyclictest0-21swapper/309:40:213
4734991914,3cyclictest33-21ksoftirqd/311:45:123
4734991910,6cyclictest0-21swapper/310:43:563
4734991910,6cyclictest0-21swapper/309:21:383
471899196,2cyclictest131rcu_preempt09:20:071
471899191,16cyclictest7457-21timerwakeupswit08:50:271
470699193,7cyclictest12-21ksoftirqd/008:10:190
470699190,5cyclictest12-21ksoftirqd/008:20:260
473499189,6cyclictest0-21swapper/311:31:083
473499189,6cyclictest0-21swapper/309:11:373
473499182,11cyclictest0-21swapper/308:25:143
473499182,10cyclictest0-21swapper/308:45:133
473499182,10cyclictest0-21swapper/308:35:133
473499182,10cyclictest0-21swapper/308:30:043
4734991812,4cyclictest33-21ksoftirqd/311:40:213
4734991810,6cyclictest0-21swapper/311:15:123
4734991810,6cyclictest0-21swapper/310:51:303
472799181,13cyclictest4814-21latency_hist07:10:012
471899185,4cyclictest19865-21sed07:25:191
471899181,14cyclictest11733-21kthreadcore07:50:181
473499179,6cyclictest0-21swapper/312:33:153
473499179,6cyclictest0-21swapper/311:24:593
473499179,6cyclictest0-21swapper/310:35:093
473499179,6cyclictest0-21swapper/310:05:573
473499179,6cyclictest0-21swapper/309:45:353
473499178,5cyclictest28366irq/58-eth007:40:143
473499177,7cyclictest0-21swapper/311:55:033
4734991711,4cyclictest0-21swapper/312:10:413
4734991711,4cyclictest0-21swapper/312:10:403
4734991711,4cyclictest0-21swapper/311:07:213
4734991711,4cyclictest0-21swapper/311:04:063
4734991711,4cyclictest0-21swapper/311:04:063
4734991711,4cyclictest0-21swapper/309:58:363
473499171,10cyclictest28366irq/58-eth010:00:123
473499171,10cyclictest0-21swapper/307:45:163
4734991710,5cyclictest33-21ksoftirqd/308:24:213
473499167,6cyclictest0-21swapper/310:30:213
473499162,9cyclictest0-21swapper/307:10:133
473499162,10cyclictest0-21swapper/307:30:133
4734991610,4cyclictest0-21swapper/310:10:203
4734991610,4cyclictest0-21swapper/310:10:203
473499157,6cyclictest0-21swapper/310:55:353
473499151,9cyclictest0-21swapper/308:40:143
473499151,9cyclictest0-21swapper/308:40:143
473499151,11cyclictest0-21swapper/308:00:193
473499151,11cyclictest0-21swapper/308:00:193
4734991510,3cyclictest0-21swapper/307:21:213
473499141,6cyclictest0-21swapper/308:15:203
471899141,10cyclictest4604-21latency_hist07:10:011
470699148,4cyclictest0-21swapper/007:10:010
473499138,3cyclictest33-21ksoftirqd/308:05:153
473499138,3cyclictest33-21ksoftirqd/307:50:273
473499138,3cyclictest0-21swapper/307:35:203
473499138,3cyclictest0-21swapper/307:35:193
473499138,3cyclictest0-21swapper/307:25:213
473499138,3cyclictest0-21swapper/307:15:293
473499132,7cyclictest0-21swapper/308:10:123
473499131,9cyclictest0-21swapper/309:05:283
473499131,9cyclictest0-21swapper/308:55:213
473499121,9cyclictest0-21swapper/309:00:183
473499121,9cyclictest0-21swapper/308:00:023
473499121,7cyclictest0-21swapper/308:55:013
473499111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional