You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 11:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Aug 31, 2026 00:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2877599583,45cyclictest8168-21ssh22:00:122
2875599581,47cyclictest15104-21ssh21:10:130
2876399564,40cyclictest17234-21ssh22:05:131
28775995110,14cyclictest131rcu_preempt23:30:122
28787994933,14cyclictest36066irq/59-eth000:15:133
28763994914,25cyclictest21-21ksoftirqd/122:20:141
28763994914,25cyclictest21-21ksoftirqd/122:20:131
28787994838,6cyclictest35966irq/58-eth023:05:133
2876399481,37cyclictest30135-21ssh22:30:161
2876399469,16cyclictest21-21ksoftirqd/100:30:151
2876399462,36cyclictest1242-21ssh22:50:121
28755994514,25cyclictest12-21ksoftirqd/023:45:140
2877599441,27cyclictest26274-21ssh22:10:152
2875599444,26cyclictest1252-21ssh22:50:130
28775994320,9cyclictest5019-21ssh23:10:122
2875599423,10cyclictest131rcu_preempt23:15:150
28755994224,11cyclictest131rcu_preempt20:15:120
2875599421,29cyclictest23113-21ssh23:20:140
28775994131,6cyclictest261rcuc/221:55:142
28775994119,16cyclictest0-21swapper/221:30:142
2878799406,26cyclictest35966irq/58-eth022:55:143
28775994020,15cyclictest0-21swapper/221:45:122
2878799394,21cyclictest35966irq/58-eth019:15:133
2878799394,21cyclictest35966irq/58-eth019:15:133
2876399391,29cyclictest14693-21diskmemload23:05:141
2878799385,20cyclictest35966irq/58-eth021:50:133
2877599384,12cyclictest27-21ksoftirqd/223:40:132
2877599382,22cyclictest930-21snmpd19:20:162
28763993817,10cyclictest21-21ksoftirqd/123:50:121
2878799371,29cyclictest35966irq/58-eth023:10:133
28755993716,18cyclictest12-21ksoftirqd/020:40:140
2877599362,17cyclictest17876-21ssh21:29:172
2877599361,20cyclictest23103-21ssh23:20:142
2875599350,5cyclictest131rcu_preempt00:30:160
2878799344,19cyclictest35966irq/58-eth021:30:133
2878799335,18cyclictest35966irq/58-eth023:20:143
28787993224,6cyclictest0-21swapper/321:20:153
2877599322,24cyclictest14197-21diskstats23:15:162
2877599321,25cyclictest751-21ssh21:20:152
2875599322,22cyclictest30106-21ssh00:16:160
2877599314,15cyclictest27346irq/40-dwc2_hso19:35:132
2877599313,23cyclictest22594-21ssh21:50:152
28755993122,6cyclictest0-21swapper/000:00:140
2878799304,18cyclictest0-21swapper/322:45:113
2877599307,17cyclictest27-21ksoftirqd/223:05:122
2877599305,16cyclictest0-21swapper/222:55:142
2877599301,22cyclictest12059-21diskmemload22:20:152
2877599301,22cyclictest12059-21diskmemload22:20:142
2876399301,24cyclictest21061-21ssh22:25:131
28755993017,8cyclictest12-21ksoftirqd/021:45:130
28755993017,10cyclictest12-21ksoftirqd/022:10:130
2878799294,13cyclictest36066irq/59-eth021:35:143
2877599295,9cyclictest20993-21sh23:55:132
2877599294,18cyclictest27346irq/40-dwc2_hso20:50:132
2877599292,20cyclictest0-21swapper/220:55:132
28775992913,8cyclictest24146-21cron00:15:002
2876399292,21cyclictest21008-21ssh23:55:141
2875599298,14cyclictest18903-21ssh21:30:130
2875599293,23cyclictest8177-21ssh22:00:120
28787992815,9cyclictest35966irq/58-eth023:25:143
2878799281,15cyclictest35966irq/58-eth022:20:143
2878799281,15cyclictest35966irq/58-eth022:20:133
2877599283,19cyclictest27346irq/40-dwc2_hso20:35:122
2877599283,19cyclictest27346irq/40-dwc2_hso20:35:112
2877599283,19cyclictest0-21swapper/200:19:162
2877599282,21cyclictest23702-21ssh22:44:162
2877599282,20cyclictest0-21swapper/220:05:132
2877599282,19cyclictest0-21swapper/219:55:142
2876399282,20cyclictest16464-21ssh23:34:171
2875599282,22cyclictest17528-21ssh23:35:150
2875599282,19cyclictest27927-21ssh21:35:140
28755992821,5cyclictest12-21ksoftirqd/022:35:120
2875599281,2cyclictest131rcu_preempt20:10:120
2875599281,20cyclictest24116-21ssh21:15:140
2878799273,18cyclictest0-21swapper/322:30:153
2878799273,16cyclictest35966irq/58-eth023:35:153
28787992715,8cyclictest35966irq/58-eth022:35:153
2877599273,13cyclictest27-21ksoftirqd/223:25:132
28775992719,5cyclictest0-21swapper/200:25:242
2877599271,22cyclictest23412-21cut20:45:162
2877599270,15cyclictest28210-21find00:35:102
2876399270,22cyclictest9893-21ssh21:25:131
2875599276,15cyclictest0-21swapper/022:25:130
2875599272,19cyclictest18018-21ssh22:21:520
2875599272,19cyclictest18018-21ssh22:21:510
2875599272,19cyclictest17356-21ssh22:05:170
2878799263,15cyclictest0-21swapper/323:54:163
2878799263,14cyclictest0-21swapper/321:10:133
28787992610,10cyclictest35966irq/58-eth000:20:143
2877599262,19cyclictest12020-21ssh23:50:112
2877599262,15cyclictest27346irq/40-dwc2_hso20:15:122
2877599261,21cyclictest4262-21ls20:25:152
28775992612,10cyclictest1268-21sh00:20:152
28775992610,6cyclictest17629-21/usr/sbin/munin22:05:182
2876399262,8cyclictest0-21swapper/121:59:161
2876399262,20cyclictest73-21kswapd023:20:151
2876399262,18cyclictest32628-21ssh00:19:161
2876399261,21cyclictest29983-21ssh00:00:141
2878799252,13cyclictest35966irq/58-eth000:10:153
2877599255,8cyclictest27-21ksoftirqd/219:40:142
2877599254,13cyclictest27-21ksoftirqd/222:33:262
28775992513,8cyclictest27-21ksoftirqd/220:40:322
28775992511,10cyclictest0-21swapper/219:10:042
2875599253,17cyclictest8572-21ssh23:30:140
2875599251,20cyclictest15588-21ssh00:10:150
28755992511,11cyclictest7342-21ssh00:05:210
28787992416,5cyclictest35966irq/58-eth022:00:123
28787992412,6cyclictest0-21swapper/320:05:133
2878799241,13cyclictest35966irq/58-eth020:15:133
2877599249,11cyclictest15485-21irqcore21:10:182
28775992414,8cyclictest0-21swapper/219:15:302
28775992414,8cyclictest0-21swapper/219:15:292
28775992413,7cyclictest27-21ksoftirqd/222:35:132
28775992412,10cyclictest0-21swapper/223:35:142
2877599241,13cyclictest11845-21ssh22:20:022
2877599241,10cyclictest2360066ssh23:00:312
2876399248,12cyclictest21-21ksoftirqd/120:50:121
2876399248,11cyclictest26530-21diskmemload23:40:141
2876399241,4cyclictest131rcu_preempt23:35:151
2876399241,19cyclictest10295-21ssh22:55:151
2876399241,16cyclictest14693-21diskmemload21:20:161
2875599248,13cyclictest12-21ksoftirqd/021:05:130
2875599244,15cyclictest12-21ksoftirqd/020:00:130
28755992415,6cyclictest28241-21ssh00:35:120
28755992415,6cyclictest26516-21ssh23:40:120
2875599240,22cyclictest10258-21ssh00:25:120
2878799235,11cyclictest35966irq/58-eth020:25:123
2878799232,14cyclictest35966irq/58-eth020:20:123
2877599232,15cyclictest0-21swapper/219:30:142
28775992314,4cyclictest27346irq/40-dwc2_hso20:00:142
28775992312,8cyclictest0-21swapper/223:45:192
2877599231,12cyclictest1460-21sort00:00:292
2877599231,11cyclictest22918-21/usr/sbin/munin00:30:282
28775992310,10cyclictest0-21swapper/220:12:162
2876399231,14cyclictest19389-21ssh23:00:121
2875599232,15cyclictest9313-21ssh22:17:170
2878799222,16cyclictest35966irq/58-eth022:50:133
2877599228,4cyclictest27-21ksoftirqd/221:36:162
2877599223,9cyclictest28664-21apt-get19:10:022
2877599223,7cyclictest131rcu_preempt20:20:152
2877599221,9cyclictest27281-21kthreadcore22:45:202
2877599221,5cyclictest0-21swapper/221:05:142
28775992212,7cyclictest0-21swapper/222:50:022
2877599221,12cyclictest22501-21kthreadcore22:25:202
2877599221,12cyclictest15596-21grep19:30:012
2877599221,11cyclictest8945-21cut20:30:172
2877599221,10cyclictest5792-21tr21:40:202
2877599221,10cyclictest30764-21sed19:45:172
2876399224,14cyclictest0-21swapper/121:35:141
28763992215,4cyclictest30467-21uptime22:10:301
2876399220,12cyclictest6204-21sshd19:20:151
2875599222,17cyclictest5047-21ssh23:10:130
2875599222,16cyclictest12933-1kworker/u9:122:45:110
28755992212,5cyclictest12-21ksoftirqd/023:55:140
2875599221,18cyclictest1269-21rm00:20:150
2878799214,13cyclictest35966irq/58-eth000:25:123
2878799212,9cyclictest35966irq/58-eth019:35:143
2877599213,8cyclictest6210-21kthreadcore21:00:182
2877599212,10cyclictest7826-21kthreadcore00:05:212
2877599211,10cyclictest24953-21kthreadcore21:15:202
2876399219,3cyclictest0-21swapper/100:07:131
28763992111,7cyclictest19279-21irqcore21:30:181
2876399211,16cyclictest447-21apt-get20:55:131
2876399211,16cyclictest32232-21ssh00:35:281
2876399211,16cyclictest2941-21ssh22:15:131
2876399211,12cyclictest24699-21ssh22:45:121
2875599217,8cyclictest0-21swapper/019:45:140
2875599215,9cyclictest15737-21ssh22:40:130
2875599211,18cyclictest32045-21rm23:25:140
2875599211,17cyclictest23468-21ls20:45:180
2875599211,16cyclictest30107-21ssh22:30:150
2878799209,9cyclictest0-21swapper/300:35:323
2878799209,6cyclictest0-21swapper/323:34:163
2878799202,9cyclictest35966irq/58-eth000:05:123
2878799202,11cyclictest35966irq/58-eth020:50:123
2878799201,13cyclictest35966irq/58-eth022:15:143
2877599201,16cyclictest3761-21kthreadcore19:50:192
2876399208,2cyclictest131rcu_preempt19:50:111
28763992014,4cyclictest0-21swapper/119:25:151
28763992012,6cyclictest0-21swapper/122:45:001
28763992012,5cyclictest24555-21kthreadcore21:50:221
28763992012,5cyclictest21-21ksoftirqd/121:20:011
2875599207,11cyclictest12-21ksoftirqd/023:05:140
2875599202,12cyclictest6646-21ssh21:21:400
2875599201,3cyclictest0-21swapper/019:30:230
28755992011,6cyclictest10390-21ssh22:55:160
28755992011,6cyclictest0-21swapper/020:50:250
2875599201,15cyclictest20749-21cut19:35:160
28755992010,6cyclictest121ksoftirqd/021:40:120
2878799197,6cyclictest0-21swapper/319:55:133
2878799193,11cyclictest0-21swapper/321:26:173
2878799192,12cyclictest35966irq/58-eth022:40:113
2878799191,12cyclictest0-21swapper/323:45:183
28787991910,5cyclictest35966irq/58-eth000:34:523
2876399199,7cyclictest26826-21cat20:45:291
2876399192,2cyclictest0-21swapper/121:10:131
2876399191,8cyclictest30419-21fschecks_time19:45:151
28763991912,4cyclictest0-21swapper/120:40:271
2876399191,16cyclictest18997-21ls00:10:241
2876399191,15cyclictest6734-21apt-get22:35:131
2875599192,3cyclictest131rcu_preempt21:00:100
2875599191,7cyclictest354-21sed19:10:280
28755991910,6cyclictest19228-21ssh23:00:020
2878799188,6cyclictest35966irq/58-eth021:40:123
2878799183,11cyclictest0-21swapper/321:45:173
2878799182,9cyclictest0-21swapper/322:05:173
2878799182,11cyclictest0-21swapper/322:11:153
2878799182,11cyclictest0-21swapper/321:18:163
2878799182,10cyclictest0-21swapper/321:05:153
2878799182,10cyclictest0-21swapper/320:55:053
2878799182,10cyclictest0-21swapper/320:10:133
2878799182,10cyclictest0-21swapper/319:40:133
2878799182,10cyclictest0-21swapper/319:10:133
28787991811,5cyclictest0-21swapper/322:25:213
2878799181,14cyclictest35966irq/58-eth023:00:133
2878799181,10cyclictest0-21swapper/321:05:013
2876399189,6cyclictest12836-21usage_EthcTask22:00:301
2876399184,9cyclictest17214-21ls20:05:141
2876399182,11cyclictest21546-21ssh21:49:161
2876399182,11cyclictest0-21swapper/119:40:141
2876399181,9cyclictest10524-21sshd00:25:171
28763991811,5cyclictest0-21swapper/123:15:181
28763991811,5cyclictest0-21swapper/119:35:171
2876399181,13cyclictest8637-21ssh23:10:241
28763991810,5cyclictest607-21klogd20:35:011
2875599189,6cyclictest32282-21ssh21:55:190
2875599187,8cyclictest12011-21apt-get23:50:110
2875599181,14cyclictest26158-21sh21:50:240
2875599181,13cyclictest28767-21apt-config19:10:020
2878799179,6cyclictest0-21swapper/323:40:053
2878799179,6cyclictest0-21swapper/321:55:183
2878799173,8cyclictest0-21swapper/323:15:163
2878799173,8cyclictest0-21swapper/320:00:133
2878799172,10cyclictest0-21swapper/320:40:133
2878799172,10cyclictest0-21swapper/320:35:133
2878799172,10cyclictest0-21swapper/320:35:133
2878799171,5cyclictest0-21swapper/300:00:153
28787991711,4cyclictest0-21swapper/323:55:293
2878799171,13cyclictest0-21swapper/319:30:273
2876399178,7cyclictest0-21swapper/121:40:161
2876399178,5cyclictest3135-21sh23:45:141
2876399173,9cyclictest0-21swapper/119:55:151
2876399171,6cyclictest5319-21apt-get21:00:111
2876399171,13cyclictest22394-21sed20:10:171
2875599179,6cyclictest7559-21sshd19:55:160
2875599176,8cyclictest991-21grep20:55:180
2875599173,12cyclictest8674-21latency_hist20:30:020
2875599172,12cyclictest13464-21ntp_192.168.11421:25:250
2875599172,10cyclictest0-21swapper/019:50:120
2875599172,10cyclictest0-21swapper/019:25:140
2876399168,6cyclictest1110-21cron19:15:011
2876399167,6cyclictest1572-21sh23:25:201
2876399166,7cyclictest11974-21grep21:05:201
2876399165,3cyclictest0-21swapper/120:35:211
2876399165,3cyclictest0-21swapper/120:35:201
2876399161,6cyclictest5893-21kthreadcore20:25:191
2876399161,12cyclictest30206-21ntp_states20:15:251
2876399161,12cyclictest16161-21unixbench_singl20:00:281
2876399161,11cyclictest4968-21ntp_states00:20:261
2876399160,13cyclictest3841-21cat20:25:011
2875599169,4cyclictest0-21swapper/020:20:290
2875599166,7cyclictest27828-21cut19:40:200
2875599166,7cyclictest12-21ksoftirqd/020:30:140
2875599162,11cyclictest19541-21kthreadcore20:05:210
2875599161,12cyclictest13778-21fschecks_time20:35:150
2875599161,12cyclictest13778-21fschecks_time20:35:140
2876399157,6cyclictest1441-21entropy19:15:151
2876399157,6cyclictest1441-21entropy19:15:141
2876399152,10cyclictest17945-21kthreadcore19:30:211
2876399150,6cyclictest28740-21cyclictest19:10:011
2875599157,6cyclictest5946-21cron19:20:020
2875599151,11cyclictest2698-21tr19:15:200
2875599151,11cyclictest2698-21tr19:15:190
2878799141,10cyclictest0-21swapper/320:45:153
2878799132,8cyclictest0-21swapper/319:20:153
2878799131,6cyclictest0-21swapper/320:30:223
2878799131,10cyclictest0-21swapper/319:50:153
2878799131,10cyclictest0-21swapper/319:25:203
2878799121,4cyclictest0-21swapper/319:45:293
2878799101,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional