You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Mar 16, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
250799614,10cyclictest12-21ksoftirqd/021:35:110
2526994712,6cyclictest131rcu_preempt22:40:122
252699462,22cyclictest15567-21kthreadcore00:05:222
2526994512,20cyclictest131rcu_preempt22:00:132
252699431,25cyclictest0-21swapper/219:30:132
2526994230,7cyclictest0-21swapper/223:45:142
253499401,21cyclictest28366irq/58-eth022:05:123
252699396,17cyclictest131rcu_preempt19:15:122
252699390,2cyclictest131rcu_preempt23:50:112
253499385,21cyclictest28366irq/58-eth021:50:123
252699382,28cyclictest5824-21cut00:00:222
252699381,21cyclictest24886-21sh00:30:142
252699372,23cyclictest30305-21ssh00:15:142
2516993611,22cyclictest21-21ksoftirqd/122:15:121
253499350,27cyclictest0-21swapper/300:35:123
250799359,4cyclictest131rcu_preempt22:50:150
2526993420,7cyclictest27-21ksoftirqd/222:45:112
2516993411,19cyclictest21-21ksoftirqd/121:55:151
2516993411,19cyclictest21-21ksoftirqd/121:55:141
250799343,15cyclictest12-21ksoftirqd/023:20:120
250799342,29cyclictest28409-21ssh21:50:130
2507993413,16cyclictest12-21ksoftirqd/021:00:110
252699331,5cyclictest131rcu_preempt22:50:132
251699337,3cyclictest131rcu_preempt00:35:141
250799336,5cyclictest131rcu_preempt21:15:130
250799336,5cyclictest131rcu_preempt21:15:130
250799332,28cyclictest1667-21ssh23:05:140
2526993218,11cyclictest27-21ksoftirqd/221:40:122
2516993215,11cyclictest0-21swapper/123:15:241
251699321,22cyclictest32104-21rm22:10:121
251699320,3cyclictest20623-21diskmemload23:30:121
253499310,23cyclictest0-21swapper/321:55:133
253499310,23cyclictest0-21swapper/321:55:133
252699318,14cyclictest19735-21ssh23:15:122
2526993110,16cyclictest12434-21rm22:35:102
250799316,3cyclictest131rcu_preempt00:25:130
2534993010,13cyclictest0-21swapper/319:50:123
2516993023,5cyclictest21-21ksoftirqd/119:50:121
253499298,9cyclictest28366irq/58-eth021:30:123
252699294,14cyclictest10657-21ssh23:10:132
252699291,11cyclictest924-21snmpd20:45:142
250799296,3cyclictest131rcu_preempt19:35:120
2507992915,5cyclictest15728-21ssh21:25:130
253499285,15cyclictest28366irq/58-eth022:50:143
253499285,13cyclictest28366irq/58-eth021:40:123
2534992815,10cyclictest0-21swapper/300:05:133
2534992811,10cyclictest0-21swapper/319:45:133
252699282,16cyclictest22346irq/40-dwc2_hso20:50:132
251699289,3cyclictest21-21ksoftirqd/122:48:091
250799286,4cyclictest131rcu_preempt21:30:130
250799284,2cyclictest131rcu_preempt19:20:120
253499273,17cyclictest0-21swapper/323:30:123
253499272,18cyclictest0-21swapper/300:10:133
252699273,15cyclictest22346irq/40-dwc2_hso21:20:122
252699272,18cyclictest8738-21ssh22:15:142
252699272,18cyclictest0-21swapper/219:35:142
252699272,14cyclictest5287-21ssh23:25:122
252699272,14cyclictest5287-21ssh23:25:112
2516992712,9cyclictest0-21swapper/100:03:221
2516992712,5cyclictest0-21swapper/121:37:151
250799279,3cyclictest12-21ksoftirqd/000:17:290
2507992713,10cyclictest151rcuc/019:10:130
250799271,21cyclictest3344-21ssh00:00:150
253499262,11cyclictest0-21swapper/300:15:133
253499261,19cyclictest0-21swapper/322:40:133
252699264,16cyclictest22346irq/40-dwc2_hso23:00:122
252699263,14cyclictest22346irq/40-dwc2_hso20:15:132
252699262,18cyclictest21323-21ssh00:10:122
2526992618,4cyclictest22346irq/40-dwc2_hso21:55:142
2526992618,4cyclictest22346irq/40-dwc2_hso21:55:132
252699261,15cyclictest10390-21ssh00:20:232
251699269,2cyclictest21-21ksoftirqd/122:29:271
2516992614,3cyclictest21-21ksoftirqd/123:05:221
2516992613,3cyclictest21-21ksoftirqd/100:27:581
250799269,11cyclictest0-21swapper/023:37:530
250799265,2cyclictest131rcu_preempt20:30:130
2507992614,3cyclictest12-21ksoftirqd/022:40:130
2507992613,10cyclictest8084-21ssh21:55:250
2507992613,10cyclictest8084-21ssh21:55:240
2507992612,4cyclictest12-21ksoftirqd/000:40:010
2507992611,3cyclictest12-21ksoftirqd/000:10:260
253499255,12cyclictest0-21swapper/323:10:123
253499253,17cyclictest28366irq/58-eth022:25:123
253499253,17cyclictest28366irq/58-eth021:25:133
253499252,17cyclictest28366irq/58-eth000:20:133
253499252,14cyclictest28366irq/58-eth023:15:133
2534992510,11cyclictest0-21swapper/323:20:213
252699252,20cyclictest0-21swapper/221:50:132
2526992516,7cyclictest531-21usb-storage22:25:122
252699251,14cyclictest15737-21sh21:25:132
251699257,3cyclictest19854-21irqcore21:45:191
2516992515,7cyclictest16214-21ssh22:35:261
2516992514,3cyclictest21-21ksoftirqd/123:22:541
2516992512,4cyclictest21-21ksoftirqd/120:45:181
2516992512,3cyclictest21-21ksoftirqd/100:20:021
2516992511,9cyclictest0-21swapper/121:31:551
2516992511,4cyclictest21-21ksoftirqd/119:40:301
2516992511,4cyclictest21-21ksoftirqd/119:40:301
2516992510,3cyclictest131rcu_preempt21:25:281
250799259,2cyclictest131rcu_preempt20:55:120
250799258,11cyclictest12-21ksoftirqd/022:20:140
250799253,18cyclictest8718-21ssh22:15:130
2507992513,3cyclictest12-21ksoftirqd/021:10:200
2507992513,3cyclictest12-21ksoftirqd/019:40:280
2507992513,3cyclictest12-21ksoftirqd/019:40:270
2507992512,3cyclictest12-21ksoftirqd/023:55:220
2507992512,3cyclictest12-21ksoftirqd/020:05:160
250799251,20cyclictest4328-21cut20:50:210
2507992510,5cyclictest0-21swapper/023:47:350
2507992510,3cyclictest12-21ksoftirqd/000:05:140
253499248,9cyclictest28366irq/58-eth023:00:133
253499242,17cyclictest0-21swapper/322:45:113
2534992417,5cyclictest321rcuc/322:20:143
253499241,13cyclictest28366irq/58-eth022:00:133
252699249,7cyclictest131rcu_preempt19:50:002
252699242,16cyclictest0-21swapper/222:30:132
252699241,13cyclictest924-21snmpd23:55:262
251699248,5cyclictest0-21swapper/120:10:201
251699248,4cyclictest17166systemd-network00:12:531
2516992413,3cyclictest21-21ksoftirqd/123:51:591
2516992413,3cyclictest21-21ksoftirqd/120:05:291
2516992413,2cyclictest21-21ksoftirqd/123:01:591
2516992413,2cyclictest21-21ksoftirqd/122:53:141
2516992412,4cyclictest21-21ksoftirqd/120:30:191
2516992412,3cyclictest21-21ksoftirqd/123:47:051
2516992412,3cyclictest21-21ksoftirqd/122:20:221
2516992411,4cyclictest21-21ksoftirqd/100:23:121
2516992411,3cyclictest21-21ksoftirqd/123:25:151
2516992411,3cyclictest21-21ksoftirqd/123:25:141
2516992411,3cyclictest21-21ksoftirqd/120:50:161
2516992410,11cyclictest15610-21ssh22:00:221
250799249,10cyclictest0-21swapper/022:38:550
250799248,11cyclictest12-21ksoftirqd/022:05:120
2507992414,2cyclictest12-21ksoftirqd/023:10:270
2507992413,3cyclictest12-21ksoftirqd/020:45:210
250799241,20cyclictest18087-21latency_hist20:00:010
2507992411,4cyclictest12-21ksoftirqd/023:50:160
2507992411,4cyclictest12-21ksoftirqd/023:00:190
2507992411,3cyclictest12-21ksoftirqd/023:30:300
2507992410,4cyclictest12-21ksoftirqd/023:44:560
2507992410,4cyclictest12-21ksoftirqd/022:55:210
2507992410,4cyclictest12-21ksoftirqd/021:22:590
2507992410,4cyclictest12-21ksoftirqd/000:20:200
2507992410,3cyclictest12-21ksoftirqd/022:47:460
2507992410,3cyclictest12-21ksoftirqd/021:49:380
253499233,13cyclictest0-21swapper/323:50:143
253499233,13cyclictest0-21swapper/300:25:133
2534992318,3cyclictest321rcuc/323:39:233
253499231,13cyclictest0-21swapper/322:10:133
252699238,11cyclictest18080-21kthreadcore22:55:222
252699237,13cyclictest17280-21grep19:55:272
252699236,3cyclictest131rcu_preempt21:35:222
252699232,18cyclictest0-21swapper/223:35:162
252699232,17cyclictest22346irq/40-dwc2_hso20:35:122
252699232,15cyclictest22346irq/40-dwc2_hso21:10:132
252699231,11cyclictest924-21snmpd23:20:132
2526992311,10cyclictest19272-21sshd20:32:212
251699239,12cyclictest21-21ksoftirqd/123:55:331
251699233,16cyclictest11760-21cut23:10:191
251699232,14cyclictest29994-21ssh21:15:111
251699232,14cyclictest29994-21ssh21:15:111
251699231,8cyclictest131rcu_preempt21:23:191
2516992313,3cyclictest21-21ksoftirqd/119:15:201
2516992312,3cyclictest21-21ksoftirqd/100:05:291
2516992312,2cyclictest21-21ksoftirqd/123:44:451
2516992311,3cyclictest21-21ksoftirqd/123:35:251
2516992311,3cyclictest21-21ksoftirqd/122:08:291
2516992311,3cyclictest21-21ksoftirqd/120:25:201
2516992310,5cyclictest0-21swapper/121:51:461
2516992310,4cyclictest21-21ksoftirqd/100:30:191
2516992310,10cyclictest7815-21sshd20:20:221
250799239,3cyclictest12-21ksoftirqd/023:15:140
2507992312,3cyclictest12-21ksoftirqd/023:25:220
2507992312,3cyclictest12-21ksoftirqd/023:25:210
2507992311,3cyclictest12-21ksoftirqd/021:41:310
2507992310,3cyclictest12-21ksoftirqd/022:30:160
2507992310,3cyclictest12-21ksoftirqd/020:15:130
253499223,10cyclictest0-21swapper/322:30:143
253499222,14cyclictest28366irq/58-eth021:10:143
252699224,9cyclictest20669-21ssh21:45:202
252699223,16cyclictest32160-21kthreadcore21:15:212
252699223,16cyclictest32160-21kthreadcore21:15:202
252699222,4cyclictest131rcu_preempt19:10:212
252699222,11cyclictest0-21swapper/200:29:532
2526992212,8cyclictest0-21swapper/223:43:142
252699221,16cyclictest12253-21apt-get19:20:112
2526992211,4cyclictest1355-21/usr/sbin/munin00:35:322
252699221,10cyclictest27446-21usage_Anybus22:05:312
251699229,3cyclictest21-21ksoftirqd/121:40:321
251699228,3cyclictest21-21ksoftirqd/120:00:171
251699226,13cyclictest21603-21ps21:10:191
251699222,9cyclictest131rcu_preempt22:41:211
251699221,18cyclictest25532-21ls19:30:271
2516992211,3cyclictest21-21ksoftirqd/122:55:291
2516992211,3cyclictest21-21ksoftirqd/119:10:281
250799229,4cyclictest12-21ksoftirqd/019:30:170
250799229,3cyclictest12-21ksoftirqd/019:45:190
250799229,2cyclictest12-21ksoftirqd/000:32:230
250799222,17cyclictest25382-21kthreadcore20:40:170
2507992211,3cyclictest12-21ksoftirqd/019:15:130
2507992210,3cyclictest12-21ksoftirqd/022:10:310
2507992210,3cyclictest12-21ksoftirqd/022:00:180
2507992210,3cyclictest12-21ksoftirqd/019:25:180
253499214,10cyclictest0-21swapper/320:40:113
253499214,10cyclictest0-21swapper/300:00:153
253499213,13cyclictest0-21swapper/323:05:143
253499212,12cyclictest28366irq/58-eth023:45:143
253499212,11cyclictest0-21swapper/323:45:023
253499212,11cyclictest0-21swapper/300:30:133
2534992110,6cyclictest28466irq/59-eth022:35:113
252699218,4cyclictest918-21basename21:35:022
252699217,10cyclictest27-21ksoftirqd/223:30:212
252699213,7cyclictest131rcu_preempt22:10:282
252699213,12cyclictest0-21swapper/219:40:132
252699213,12cyclictest0-21swapper/219:40:122
252699212,3cyclictest131rcu_preempt19:25:212
252699212,2cyclictest131rcu_preempt21:00:142
251699218,8cyclictest21-21ksoftirqd/122:31:121
2516992111,3cyclictest21-21ksoftirqd/119:35:291
2516992110,3cyclictest21-21ksoftirqd/121:05:111
2516992110,3cyclictest21-21ksoftirqd/120:55:111
250799219,3cyclictest12-21ksoftirqd/020:25:160
250799219,2cyclictest12-21ksoftirqd/020:00:270
250799218,3cyclictest12-21ksoftirqd/020:35:130
250799218,3cyclictest12-21ksoftirqd/019:50:130
2507992111,2cyclictest12-21ksoftirqd/022:25:210
2507992110,3cyclictest12-21ksoftirqd/020:20:170
253499203,11cyclictest28366irq/58-eth023:25:123
253499203,11cyclictest28366irq/58-eth023:25:123
253499202,13cyclictest28366irq/58-eth021:15:253
253499202,13cyclictest28366irq/58-eth021:15:243
2534992015,3cyclictest321rcuc/319:20:213
252699202,15cyclictest24492-21fschecks_time20:40:142
252699201,9cyclictest6949-21irqrtprio20:55:152
252699201,9cyclictest5353-21fschecks_time20:20:162
252699201,9cyclictest12223-21grep19:50:282
252699201,16cyclictest12170-21kthreadcore20:25:212
252699201,15cyclictest1903-21fschecks_time23:05:172
252699201,11cyclictest2616-21latency_hist19:10:022
252699201,10cyclictest17982-21fschecks_time22:20:172
251699209,2cyclictest21-21ksoftirqd/120:35:191
251699203,15cyclictest11407-21fschecks_count21:00:141
251699201,4cyclictest131rcu_preempt19:45:281
2516992010,2cyclictest21-21ksoftirqd/119:25:251
250799205,3cyclictest131rcu_preempt21:05:200
2507992010,3cyclictest12-21ksoftirqd/020:10:160
253499192,11cyclictest0-21swapper/320:25:153
2534991915,3cyclictest321rcuc/323:57:153
253499191,12cyclictest0-21swapper/320:55:143
252699196,10cyclictest20724-21ssh21:10:022
252699192,9cyclictest28914-21kthreadcore20:10:202
252699191,9cyclictest22162-21grep20:00:312
252699191,15cyclictest24157-21kthreadcore20:05:182
251699199,3cyclictest21-21ksoftirqd/119:10:011
251699199,2cyclictest21-21ksoftirqd/119:20:151
251699198,3cyclictest21-21ksoftirqd/119:55:281
2516991913,4cyclictest0-21swapper/120:15:221
253499189,6cyclictest0-21swapper/321:20:263
253499182,11cyclictest0-21swapper/321:35:133
253499182,11cyclictest0-21swapper/319:35:143
2534991813,3cyclictest321rcuc/322:58:563
2534991813,3cyclictest321rcuc/322:18:063
2534991813,3cyclictest321rcuc/319:40:303
2534991813,3cyclictest321rcuc/319:40:293
2534991812,4cyclictest321rcuc/319:59:223
2534991810,6cyclictest0-21swapper/321:45:153
250799181,15cyclictest2481-21cyclictest19:10:010
253499176,8cyclictest321rcuc/319:25:133
253499172,10cyclictest0-21swapper/320:05:143
2534991712,3cyclictest0-21swapper/320:10:153
253499171,12cyclictest0-21swapper/320:35:153
253499171,10cyclictest0-21swapper/320:50:133
251699173,12cyclictest24820-21sed20:40:171
253499162,9cyclictest0-21swapper/320:20:143
253499162,10cyclictest0-21swapper/319:30:143
2534991611,3cyclictest0-21swapper/321:00:263
2534991611,3cyclictest0-21swapper/320:30:293
2534991611,3cyclictest0-21swapper/320:03:223
2534991611,3cyclictest0-21swapper/319:15:163
253499161,13cyclictest0-21swapper/319:10:163
253499157,4cyclictest321rcuc/320:15:153
2534991510,3cyclictest0-21swapper/320:45:153
253499149,3cyclictest0-21swapper/321:05:133
253499101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional