You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-13 - 08:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Fri Mar 13, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2971099602,40cyclictest27911-21ssh22:10:121
2973799511,4cyclictest0-21swapper/221:50:122
29703994818,14cyclictest15477-21diskmemload23:45:190
29703994618,12cyclictest14786-21apt-get23:50:130
2976599455,23cyclictest28366irq/58-eth000:30:143
2976599414,22cyclictest28366irq/58-eth022:20:123
2973799412,21cyclictest14030-21kthreadcore00:25:232
2973799410,30cyclictest8509-21ssh22:35:122
29703994118,7cyclictest23561-21ssh21:13:490
29703994117,9cyclictest2026-21ssh22:12:330
2971099405,27cyclictest4508-21ssh22:15:121
2970399404,31cyclictest28960-21ssh21:35:130
2970399404,31cyclictest28960-21ssh21:35:120
29703994026,10cyclictest0-21swapper/023:05:110
29703994016,9cyclictest15477-21diskmemload23:03:280
29703994016,14cyclictest1960-21ssh23:25:140
29703994016,14cyclictest1960-21ssh23:25:130
29703994014,11cyclictest8410-21ssh21:58:510
2976599391,23cyclictest28366irq/58-eth023:25:143
2976599391,23cyclictest28366irq/58-eth023:25:133
2971099391,30cyclictest9496-21ssh00:05:141
2970399394,30cyclictest18523-21ssh00:10:130
29703993917,9cyclictest15477-21diskmemload21:32:080
29703993916,8cyclictest7502-21ssh21:21:390
2976599386,16cyclictest28366irq/58-eth019:25:123
2971099382,25cyclictest11005-21ssh23:30:121
29703993817,7cyclictest15477-21diskmemload22:47:160
29703993817,7cyclictest15477-21diskmemload22:26:120
29703993817,7cyclictest15477-21diskmemload00:32:580
29703993816,9cyclictest32695-21ssh21:18:540
29703993815,8cyclictest11231-21ssh00:23:050
2970399381,31cyclictest27534-21ssh00:15:130
29765993714,13cyclictest28366irq/58-eth000:20:113
2973799372,18cyclictest26494-21ssh23:37:232
2973799371,18cyclictest24947-21ssh21:15:122
29703993718,7cyclictest30727-21ssh23:21:110
29703993717,7cyclictest29578-21apt-get19:10:040
29703993717,7cyclictest15477-21diskmemload23:33:140
29703993717,6cyclictest15477-21diskmemload22:16:070
29703993716,8cyclictest32077-21ssh21:54:160
29703993716,8cyclictest18790-21ssh00:26:180
29703993715,8cyclictest15477-21diskmemload23:39:260
29703993715,7cyclictest15477-21diskmemload22:36:080
29703993714,10cyclictest20719-21ssh21:46:430
29703993713,7cyclictest31926-21ssh22:30:130
2970399370,27cyclictest5669-21ssh21:40:120
2976599364,19cyclictest28366irq/58-eth020:20:143
29765993613,15cyclictest28366irq/58-eth022:00:123
2976599361,21cyclictest28366irq/58-eth019:30:133
29703993622,11cyclictest12356-21apt-get22:55:150
29703993622,11cyclictest12356-21apt-get22:55:140
29703993616,8cyclictest15477-21diskmemload23:19:210
29703993615,8cyclictest15477-21diskmemload22:41:520
29703993615,7cyclictest15477-21diskmemload21:27:440
29703993613,8cyclictest27327-21ssh23:55:230
29703993612,8cyclictest15477-21diskmemload00:09:480
29703993612,8cyclictest15477-21diskmemload00:00:500
29703993612,8cyclictest15477-21diskmemload00:00:490
29765993521,5cyclictest0-21swapper/320:45:123
29703993528,3cyclictest27507-21apt-get20:15:130
29703993518,8cyclictest5942-21/usr/sbin/munin19:15:270
29703993516,7cyclictest15343-21ssh23:14:290
29703993514,8cyclictest15477-21diskmemload23:43:570
2976599342,21cyclictest28366irq/58-eth023:40:123
29765993417,12cyclictest28366irq/58-eth021:10:153
2973799340,18cyclictest27949-21kthreadcore23:20:212
2971099341,24cyclictest370-21ssh00:00:121
2971099341,24cyclictest370-21ssh00:00:111
29710993410,10cyclictest29683-21cyclictest20:50:131
29703993416,12cyclictest0-21swapper/019:45:110
2976599336,13cyclictest28366irq/58-eth022:55:133
2976599336,13cyclictest28366irq/58-eth022:55:123
2976599331,20cyclictest28366irq/58-eth022:50:123
29737993320,7cyclictest261rcuc/221:10:132
2970399332,5cyclictest131rcu_preempt20:30:140
29703993315,12cyclictest0-21swapper/022:50:120
29703993314,8cyclictest21755-21if_eth119:35:170
29703993314,6cyclictest32312-21apt-get20:20:150
29703993313,10cyclictest18697-21ssh22:20:450
2973799328,18cyclictest131rcu_preempt23:00:112
2973799325,2cyclictest131rcu_preempt00:10:122
29737993219,6cyclictest22346irq/40-dwc2_hso19:45:132
2971099321,22cyclictest13651-21ssh22:20:121
29703993221,8cyclictest16681-21apt-get19:30:120
29703993215,12cyclictest0-21swapper/019:20:110
29703993211,8cyclictest28535-21apt-get20:50:020
29703993211,6cyclictest5674-21ssh00:37:400
2976599311,21cyclictest28366irq/58-eth021:15:113
2976599311,16cyclictest28366irq/58-eth023:30:133
2973799314,19cyclictest27-21ksoftirqd/222:40:262
2973799312,19cyclictest27-21ksoftirqd/219:55:202
29703993113,7cyclictest23794-21ssh22:05:330
2976599301,14cyclictest28366irq/58-eth023:35:113
2973799307,7cyclictest131rcu_preempt23:44:552
2973799305,17cyclictest22346irq/40-dwc2_hso22:55:122
2973799305,17cyclictest22346irq/40-dwc2_hso22:55:122
2971099307,3cyclictest131rcu_preempt23:05:241
2971099303,5cyclictest21-21ksoftirqd/122:55:171
2971099303,5cyclictest21-21ksoftirqd/122:55:161
2971099303,19cyclictest680-21ssh21:55:131
2970399303,4cyclictest131rcu_preempt20:10:100
29703993021,7cyclictest22754-21sendmail_mailtr20:40:250
29703993018,9cyclictest24291-21interrupts20:45:170
29703993012,8cyclictest21794-21usage_Anybus20:05:290
29703993011,7cyclictest16572-21ssh22:02:450
2976599295,11cyclictest28366irq/58-eth022:10:123
2973799298,13cyclictest27-21ksoftirqd/221:48:582
2973799297,7cyclictest27-21ksoftirqd/223:05:172
2973799296,6cyclictest27-21ksoftirqd/221:38:542
2973799296,6cyclictest27-21ksoftirqd/221:38:532
2973799293,3cyclictest131rcu_preempt22:20:112
29737992916,7cyclictest27-21ksoftirqd/200:05:152
2973799291,16cyclictest10810-21ssh21:25:142
2971099297,11cyclictest21-21ksoftirqd/122:30:161
2971099295,3cyclictest131rcu_preempt21:10:291
2971099293,7cyclictest21-21ksoftirqd/122:10:011
2971099292,21cyclictest8875-21ssh23:27:581
2971099292,21cyclictest8875-21ssh23:27:571
2971099291,20cyclictest29186-21ssh23:40:121
29703992920,6cyclictest1212-21apt-get20:55:130
29703992920,6cyclictest1212-21apt-get20:55:130
2976599285,10cyclictest28366irq/58-eth023:55:123
2973799285,16cyclictest27-21ksoftirqd/222:50:202
2973799283,17cyclictest18927-21latency_hist20:40:002
2973799282,15cyclictest14808-21ssh23:50:132
29737992812,8cyclictest27-21ksoftirqd/200:17:192
2973799281,15cyclictest27-21ksoftirqd/200:35:292
2971099285,5cyclictest21-21ksoftirqd/121:35:231
2971099285,5cyclictest21-21ksoftirqd/121:35:221
2971099282,19cyclictest14031-21kthreadcore00:25:221
29710992810,9cyclictest29683-21cyclictest20:55:131
29710992810,9cyclictest29683-21cyclictest20:55:121
29703992820,6cyclictest9239-21sshd20:28:200
29703992818,7cyclictest14707-21/usr/sbin/munin21:05:280
2970399281,11cyclictest16302-21kthreadcore20:35:210
2976599272,17cyclictest28366irq/58-eth022:45:133
2973799279,11cyclictest27-21ksoftirqd/222:28:412
2973799278,12cyclictest131rcu_preempt22:05:302
2973799277,8cyclictest131rcu_preempt20:20:132
2973799275,12cyclictest22346irq/40-dwc2_hso20:00:112
2973799274,14cyclictest27-21ksoftirqd/221:20:192
2973799274,12cyclictest27-21ksoftirqd/222:00:012
2973799271,17cyclictest20352-21ls19:30:272
2973799271,10cyclictest131rcu_preempt20:50:202
29737992710,11cyclictest27-21ksoftirqd/221:58:522
2971099278,11cyclictest9650irq/52-mmc019:50:121
2971099277,10cyclictest21-21ksoftirqd/123:20:231
2971099276,10cyclictest21-21ksoftirqd/121:05:181
2970399275,13cyclictest0-21swapper/021:00:120
2976599263,17cyclictest0-21swapper/300:28:043
2976599262,12cyclictest28366irq/58-eth023:05:143
29765992615,6cyclictest28366irq/58-eth000:35:143
2973799269,11cyclictest131rcu_preempt22:18:312
2973799267,13cyclictest27-21ksoftirqd/222:45:062
2973799265,15cyclictest27-21ksoftirqd/223:25:272
2973799265,15cyclictest27-21ksoftirqd/223:25:272
2973799265,10cyclictest22346irq/40-dwc2_hso19:35:122
2973799264,12cyclictest27-21ksoftirqd/222:10:362
2973799264,12cyclictest27-21ksoftirqd/220:35:032
2973799263,13cyclictest131rcu_preempt23:55:232
29737992612,7cyclictest134-21systemd-journal21:45:002
2973799261,18cyclictest27-21ksoftirqd/220:55:182
2973799261,18cyclictest27-21ksoftirqd/220:55:172
2971099269,8cyclictest21-21ksoftirqd/120:10:141
2971099268,4cyclictest131rcu_preempt19:20:231
2971099265,10cyclictest21-21ksoftirqd/123:40:021
2971099264,5cyclictest21-21ksoftirqd/119:30:251
29703992618,6cyclictest3470-21apt-get19:50:130
2976599253,14cyclictest0-21swapper/323:20:213
2976599252,16cyclictest28366irq/58-eth022:25:123
2976599252,12cyclictest28366irq/58-eth020:50:133
2973799259,8cyclictest27-21ksoftirqd/222:30:322
2973799259,7cyclictest27-21ksoftirqd/223:30:082
2973799256,11cyclictest27-21ksoftirqd/223:16:522
2973799256,11cyclictest27-21ksoftirqd/200:30:192
2973799254,7cyclictest27-21ksoftirqd/220:15:172
2973799254,11cyclictest27-21ksoftirqd/221:34:012
2973799253,16cyclictest27-21ksoftirqd/219:50:242
2973799251,16cyclictest27-21ksoftirqd/219:10:172
29737992510,8cyclictest27-21ksoftirqd/223:10:262
2971099257,8cyclictest131rcu_preempt21:18:431
2971099257,7cyclictest21-21ksoftirqd/123:45:371
2971099257,4cyclictest131rcu_preempt21:30:221
2971099256,3cyclictest131rcu_preempt21:45:121
2971099255,7cyclictest21-21ksoftirqd/122:45:181
2971099253,5cyclictest21-21ksoftirqd/121:50:181
2971099253,10cyclictest21-21ksoftirqd/123:55:131
2971099251,8cyclictest26495-21ntp_192.168.11422:25:271
29710992518,5cyclictest201rcuc/122:35:221
2971099251,4cyclictest21-21ksoftirqd/122:00:141
29710992510,8cyclictest131rcu_preempt00:10:231
2970399250,10cyclictest16050-21cat19:25:300
29765992410,10cyclictest33-21ksoftirqd/320:30:133
2973799247,11cyclictest27-21ksoftirqd/200:21:432
2973799246,11cyclictest12543-21cut19:25:202
29737992413,3cyclictest27-21ksoftirqd/223:45:202
2973799241,16cyclictest5311-21vmstat00:00:332
2973799241,16cyclictest5311-21vmstat00:00:332
2973799241,10cyclictest0-21swapper/220:45:242
2973799240,15cyclictest131rcu_preempt20:10:102
2971099249,4cyclictest21-21ksoftirqd/123:10:271
2971099246,8cyclictest21-21ksoftirqd/119:20:011
2971099246,3cyclictest131rcu_preempt23:03:271
2971099245,3cyclictest0-21swapper/100:15:161
2971099244,11cyclictest21-21ksoftirqd/121:00:221
2971099244,11cyclictest21-21ksoftirqd/120:35:241
2971099244,10cyclictest131rcu_preempt20:00:131
2971099243,6cyclictest21-21ksoftirqd/100:32:541
2971099243,6cyclictest21-21ksoftirqd/100:20:241
2971099243,4cyclictest21-21ksoftirqd/100:35:301
2971099242,4cyclictest21-21ksoftirqd/121:25:331
29710992418,4cyclictest201rcuc/122:54:031
29710992418,4cyclictest201rcuc/122:40:271
29710992411,5cyclictest21-21ksoftirqd/123:50:191
29703992416,6cyclictest30332-21usage_Anybus19:40:290
29703992416,6cyclictest13108-21apt-get20:00:120
29703992413,8cyclictest8460-21fschecks_time19:55:160
29737992310,4cyclictest27-21ksoftirqd/220:05:172
2973799230,20cyclictest5918-21ls19:15:282
2971099236,7cyclictest21-21ksoftirqd/121:43:581
2971099236,5cyclictest21-21ksoftirqd/120:15:191
2971099236,5cyclictest21-21ksoftirqd/119:45:221
2971099234,3cyclictest131rcu_preempt21:20:141
2971099234,3cyclictest131rcu_preempt19:35:251
2971099233,7cyclictest21-21ksoftirqd/120:40:211
2971099232,14cyclictest30059-21cut19:10:161
2971099231,4cyclictest21-21ksoftirqd/120:20:181
29710992311,8cyclictest26264-21apt-get19:40:131
2971099231,13cyclictest21-21ksoftirqd/123:17:211
2971099231,12cyclictest5142-21sed20:25:191
2973799229,7cyclictest27-21ksoftirqd/220:40:202
2973799225,8cyclictest27-21ksoftirqd/220:25:222
2973799221,7cyclictest0-21swapper/221:00:112
2973799221,15cyclictest0-21swapper/219:43:192
2973799221,12cyclictest131rcu_preempt21:10:022
2971099224,3cyclictest131rcu_preempt20:30:171
2976599214,13cyclictest28366irq/58-eth023:15:133
2976599212,11cyclictest28366irq/58-eth021:45:113
2973799216,5cyclictest27-21ksoftirqd/219:10:022
2973799211,10cyclictest27-21ksoftirqd/219:20:172
2971099215,7cyclictest21-21ksoftirqd/120:45:181
2971099214,3cyclictest131rcu_preempt19:55:191
2971099212,5cyclictest21-21ksoftirqd/120:05:231
2976599207,7cyclictest28366irq/58-eth023:50:143
2976599205,8cyclictest28366irq/58-eth000:10:123
2976599205,6cyclictest0-21swapper/319:45:113
2976599202,13cyclictest0-21swapper/321:00:173
29765992010,7cyclictest0-21swapper/321:27:463
2971099205,3cyclictest131rcu_preempt19:25:251
2971099204,5cyclictest21-21ksoftirqd/119:10:011
2970399200,8cyclictest29800-21cut19:10:010
2976599199,8cyclictest0-21swapper/321:59:373
2976599199,6cyclictest28366irq/58-eth022:15:123
2976599193,11cyclictest0-21swapper/321:20:153
2976599191,14cyclictest0-21swapper/300:00:143
2976599191,14cyclictest0-21swapper/300:00:133
2976599191,12cyclictest0-21swapper/319:55:023
2976599191,11cyclictest0-21swapper/319:35:103
29765991910,7cyclictest0-21swapper/322:40:023
29765991910,6cyclictest0-21swapper/322:42:173
29765991910,6cyclictest0-21swapper/321:42:003
29765991910,6cyclictest0-21swapper/321:32:203
2976599188,8cyclictest0-21swapper/323:03:343
2976599187,8cyclictest0-21swapper/322:31:273
2976599183,10cyclictest28366irq/58-eth023:45:133
2976599182,10cyclictest0-21swapper/320:15:133
2976599182,10cyclictest0-21swapper/319:20:123
2976599181,11cyclictest0-21swapper/322:05:153
29765991810,5cyclictest28366irq/58-eth000:05:133
2976599179,6cyclictest0-21swapper/323:13:173
2976599179,6cyclictest0-21swapper/321:52:303
2976599179,3cyclictest0-21swapper/300:19:513
2976599172,10cyclictest0-21swapper/320:05:133
2976599172,10cyclictest0-21swapper/319:10:053
2976599171,13cyclictest0-21swapper/321:35:173
2976599171,13cyclictest0-21swapper/321:35:173
2976599162,9cyclictest0-21swapper/320:55:153
2976599162,9cyclictest0-21swapper/320:55:143
2976599162,10cyclictest0-21swapper/319:55:143
2976599161,10cyclictest0-21swapper/320:40:163
2976599151,12cyclictest0-21swapper/320:10:263
2976599149,3cyclictest0-21swapper/319:15:183
2976599142,10cyclictest0-21swapper/319:40:143
2976599141,7cyclictest0-21swapper/321:05:143
2976599132,9cyclictest0-21swapper/320:25:143
2976599131,5cyclictest0-21swapper/320:35:203
2976599131,10cyclictest0-21swapper/320:00:203
2976599111,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional