You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 03:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Fri Jun 05, 2026 00:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1698799522,39cyclictest5631-21ssh23:00:122
1698799502,6cyclictest0-21swapper/220:25:122
1698799502,42cyclictest0-21swapper/223:35:132
1698799491,3cyclictest0-21swapper/220:50:132
1698799491,3cyclictest0-21swapper/220:10:112
1696199493,30cyclictest13442-21ssh00:35:130
1698799486,27cyclictest2880-21diskmemload23:25:552
1698799481,2cyclictest0-21swapper/221:25:142
1698799471,7cyclictest0-21swapper/222:40:132
1698799471,4cyclictest0-21swapper/219:45:122
1698799471,43cyclictest0-21swapper/220:05:122
1698799462,40cyclictest0-21swapper/221:45:142
1698799461,2cyclictest32345-21apt-get23:15:122
1698799461,2cyclictest0-21swapper/222:15:122
1698799453,38cyclictest0-21swapper/222:25:132
1698799451,41cyclictest23833-21ssh21:39:282
1698799442,34cyclictest0-21swapper/223:05:132
1698799441,2cyclictest0-21swapper/223:20:132
1696199441,35cyclictest27976-21ssh00:25:140
1698799434,27cyclictest21001-21ssh23:45:142
1698799432,37cyclictest0-21swapper/221:15:142
1698799423,33cyclictest28258-21apt-get22:00:162
1698799421,3cyclictest0-21swapper/222:55:132
1698799421,37cyclictest0-21swapper/200:25:142
1698799421,2cyclictest3985-21apt-get19:30:122
1698799421,2cyclictest3265-21apt-get21:10:132
1698799421,2cyclictest3265-21apt-get21:10:132
1698799421,2cyclictest0-21swapper/221:40:132
1698799412,3cyclictest0-21swapper/219:50:142
1698799411,36cyclictest0-21swapper/220:40:112
1698799401,3cyclictest0-21swapper/220:55:102
1698799401,3cyclictest0-21swapper/219:25:122
16995993932,4cyclictest35966irq/58-eth023:05:133
1698799392,34cyclictest26898-21idleruntime23:30:162
1698799390,4cyclictest4755-21apt-get22:05:122
1698799383,31cyclictest0-21swapper/200:10:112
1698799382,20cyclictest1962-21cat23:50:302
1698799381,5cyclictest27538-21kthreadcore22:35:212
1698799381,4cyclictest0-21swapper/220:20:132
1698799381,4cyclictest0-21swapper/220:20:122
1698799381,34cyclictest0-21swapper/219:35:112
1698799381,12cyclictest8101-21ntp_192.168.11400:30:232
1698799380,2cyclictest0-21swapper/221:30:132
1698799373,2cyclictest0-21swapper/222:10:122
1698799371,3cyclictest6515-21apt-get23:55:132
1698799371,2cyclictest20141-21apt-get22:50:132
1698799371,2cyclictest20141-21apt-get22:50:132
1698799370,4cyclictest17178-21apt-get19:10:142
1698799361,4cyclictest15477-21wc23:40:222
1698799361,32cyclictest0-21swapper/200:20:132
1698799361,32cyclictest0-21swapper/200:20:132
1698799360,6cyclictest15437-21/usr/sbin/munin00:00:172
1697699361,27cyclictest9961-21ssh00:15:131
16995993514,16cyclictest0-21swapper/321:25:153
1698799351,6cyclictest27715-21kthreadcore00:05:252
1698799351,6cyclictest27715-21kthreadcore00:05:252
1698799351,6cyclictest24655-21tr21:20:242
1698799351,6cyclictest1797-21cut20:00:202
1698799351,5cyclictest0-21swapper/220:45:122
1698799351,3cyclictest0-21swapper/223:10:392
1698799351,2cyclictest0-21swapper/222:20:112
1698799350,5cyclictest30500-21sed19:20:282
1698799350,5cyclictest15695-21rm21:51:112
1696199356,22cyclictest664-21ntpd21:00:120
1696199354,24cyclictest4766-21ssh22:05:130
16961993513,16cyclictest10665-21timerandwakeup21:30:280
1696199351,13cyclictest12-21ksoftirqd/020:45:130
1698799346,24cyclictest16188-21tr20:15:182
1698799341,4cyclictest0-21swapper/219:55:122
1698799341,30cyclictest0-21swapper/222:31:192
1698799341,30cyclictest0-21swapper/219:40:122
1698799340,5cyclictest15971-21usage_EthcTask22:45:322
1696199343,20cyclictest12-21ksoftirqd/022:40:120
1698799331,27cyclictest0-21swapper/221:00:122
1698799330,30cyclictest1654-21sshd20:32:232
1697699331,25cyclictest24496-21ssh00:05:161
1697699331,25cyclictest24496-21ssh00:05:161
1699599325,12cyclictest0-21swapper/323:10:123
1699599320,17cyclictest35966irq/58-eth022:50:133
1699599320,17cyclictest35966irq/58-eth022:50:133
1698799321,28cyclictest19263-21/usr/sbin/munin21:55:142
1698799321,28cyclictest19263-21/usr/sbin/munin21:55:132
1698799321,28cyclictest1900-21apt20:35:122
1698799320,7cyclictest18011-21ssh00:19:272
1697699324,25cyclictest0-21swapper/121:20:141
1697699322,26cyclictest30004-21ssh23:50:131
16976993220,8cyclictest22540-21apt-get22:15:111
1696199325,6cyclictest12-21ksoftirqd/021:50:120
1696199322,24cyclictest30629-21cat23:30:280
1696199322,22cyclictest10886-21cat23:55:310
1699599312,25cyclictest35966irq/58-eth023:20:133
1699599311,20cyclictest35966irq/58-eth023:50:133
1698799311,5cyclictest24810-21kthreadcore19:15:202
1698799311,3cyclictest0-21swapper/221:05:132
1698799311,3cyclictest0-21swapper/200:35:322
1697699312,23cyclictest13143-21ssh23:57:381
16976993116,11cyclictest25868-21apt-get22:35:131
1696199316,7cyclictest12-21ksoftirqd/023:40:110
16995993013,10cyclictest0-21swapper/322:00:143
16976993020,7cyclictest11240-21ssh22:45:131
1696199302,22cyclictest22574-21sh23:09:290
1696199302,20cyclictest20151-21ssh22:50:130
1696199302,20cyclictest20151-21ssh22:50:130
16995992913,12cyclictest0-21swapper/322:15:123
16995992912,14cyclictest0-21swapper/323:35:133
16976992921,6cyclictest21-21ksoftirqd/123:40:121
16976992921,4cyclictest5356-21apt-get20:05:131
1696199292,20cyclictest16982-21ssh22:30:140
1696199292,17cyclictest628-21tcf-agent21:35:130
1696199291,22cyclictest18922-21ssh00:20:150
1696199291,22cyclictest18922-21ssh00:20:140
1699599284,16cyclictest0-21swapper/323:45:143
1699599281,17cyclictest35966irq/58-eth000:00:123
1697699285,21cyclictest8858-21ssh23:20:121
1697699284,21cyclictest12219-21ssh21:15:121
1697699283,11cyclictest32651-21kworker/u8:222:40:381
16976992815,10cyclictest26790-21apt-get19:20:131
16976992813,9cyclictest4454-21apt-get00:30:111
16976992813,10cyclictest10192-21apt-get20:10:111
1697699281,24cyclictest0-21swapper/120:15:121
1697699281,21cyclictest20171-21ssh22:50:151
1697699281,21cyclictest20171-21ssh22:50:151
16976992812,12cyclictest0-21swapper/121:25:141
1696199281,20cyclictest12-21ksoftirqd/019:30:120
16995992717,8cyclictest0-21swapper/300:10:113
1697699276,12cyclictest0-21swapper/123:36:481
1697699275,4cyclictest15597-21df_inode00:00:151
1697699275,18cyclictest27991-21ssh00:25:151
1697699274,5cyclictest21-21ksoftirqd/122:10:221
1697699274,5cyclictest21-21ksoftirqd/121:45:251
1697699274,4cyclictest201rcuc/123:13:181
1697699272,23cyclictest5637-21sh23:00:131
1697699272,19cyclictest10698-21cat21:30:281
16976992719,4cyclictest13652-21apt-get19:40:121
16976992718,5cyclictest16293-21apt-get20:50:141
1697699271,10cyclictest8059-21ssh23:19:311
1696199279,9cyclictest12-21ksoftirqd/021:40:050
16961992710,7cyclictest12-21ksoftirqd/019:20:280
1699599263,16cyclictest36066irq/59-eth021:20:143
1699599262,19cyclictest35966irq/58-eth022:45:133
16995992619,2cyclictest0-21swapper/300:25:133
1699599261,5cyclictest0-21swapper/321:40:133
16995992613,7cyclictest35966irq/58-eth021:35:123
1698799261,5cyclictest16985-21cat19:10:012
1697699267,4cyclictest201rcuc/122:57:141
1697699262,5cyclictest21-21ksoftirqd/121:14:121
1697699262,5cyclictest21-21ksoftirqd/121:14:111
1697699262,20cyclictest24699-21sh21:40:141
16976992620,3cyclictest18441-21apt-get19:45:121
16976992619,3cyclictest31572-21apt-get19:25:121
1696199263,18cyclictest31544-21rm22:20:120
1696199262,16cyclictest17775-21ssh23:25:110
1696199261,5cyclictest131rcu_preempt00:05:150
1696199261,5cyclictest131rcu_preempt00:05:150
16995992519,3cyclictest0-21swapper/320:25:123
1697699255,4cyclictest201rcuc/123:46:411
1697699255,4cyclictest201rcuc/122:29:201
1697699254,5cyclictest21-21ksoftirqd/123:28:361
1697699254,4cyclictest201rcuc/122:07:321
1697699253,6cyclictest21-21ksoftirqd/122:02:251
1697699253,5cyclictest21-21ksoftirqd/122:20:221
1697699253,5cyclictest21-21ksoftirqd/120:40:011
16976992517,5cyclictest15684-21apt-get21:35:131
1696199259,3cyclictest12-21ksoftirqd/020:00:240
1696199258,10cyclictest12-21ksoftirqd/023:35:300
1696199254,14cyclictest630-21Async09:36:310
1696199254,14cyclictest630-21Async09:36:310
1696199253,8cyclictest12-21ksoftirqd/000:16:300
1696199252,12cyclictest32373-21ssh23:15:140
1696199251,6cyclictest12-21ksoftirqd/023:00:160
1696199251,6cyclictest12-21ksoftirqd/022:35:290
16961992512,3cyclictest0-21swapper/000:34:520
16961992511,8cyclictest12-21ksoftirqd/000:00:330
1699599243,14cyclictest0-21swapper/300:20:293
1699599243,14cyclictest0-21swapper/300:20:293
16995992417,3cyclictest0-21swapper/323:15:133
16995992417,3cyclictest0-21swapper/321:30:133
16995992412,7cyclictest0-21swapper/322:05:143
1697699246,5cyclictest21-21ksoftirqd/120:25:211
1697699244,5cyclictest23085-21ssh21:55:291
1697699244,5cyclictest23085-21ssh21:55:281
1697699244,5cyclictest21-21ksoftirqd/100:20:051
1697699244,5cyclictest21-21ksoftirqd/100:20:051
1697699243,6cyclictest21-21ksoftirqd/100:35:331
1697699243,5cyclictest21-21ksoftirqd/121:50:171
1697699243,4cyclictest201rcuc/123:30:041
1697699243,12cyclictest0-21swapper/119:15:111
16976992414,7cyclictest29514-21apt-get20:30:121
1697699241,20cyclictest14543-21ssh23:05:131
1696199247,3cyclictest131rcu_preempt23:45:150
1696199247,3cyclictest131rcu_preempt19:55:120
1696199246,9cyclictest12-21ksoftirqd/022:15:110
1696199242,3cyclictest12-21ksoftirqd/019:25:110
1696199241,8cyclictest7325-21unixbench_singl21:10:300
1696199241,8cyclictest7325-21unixbench_singl21:10:290
16961992411,8cyclictest12-21ksoftirqd/021:16:300
16961992411,10cyclictest4364-21cron22:05:000
16995992311,9cyclictest0-21swapper/321:50:123
1697699236,5cyclictest0-21swapper/120:45:301
1697699235,5cyclictest21-21ksoftirqd/119:35:131
1697699235,5cyclictest21-21ksoftirqd/100:10:231
1697699233,6cyclictest21945-21kthreadcore20:55:171
1697699232,3cyclictest21-21ksoftirqd/122:32:581
1696199239,6cyclictest12-21ksoftirqd/021:46:420
1696199238,8cyclictest12-21ksoftirqd/022:49:440
1696199233,9cyclictest131rcu_preempt20:30:280
1696199231,9cyclictest131rcu_preempt23:50:300
16961992314,6cyclictest29135-21ssh22:55:140
1699599223,13cyclictest0-21swapper/323:57:383
16995992216,3cyclictest0-21swapper/320:55:103
16995992216,3cyclictest0-21swapper/319:35:113
16995992215,3cyclictest0-21swapper/319:40:123
16995992214,5cyclictest0-21swapper/319:45:123
16995992213,6cyclictest0-21swapper/323:30:123
16995992210,6cyclictest0-21swapper/322:40:213
16995992210,5cyclictest0-21swapper/300:05:153
16995992210,5cyclictest0-21swapper/300:05:153
1697699225,5cyclictest21-21ksoftirqd/120:40:021
16976992214,4cyclictest23246-21apt-get19:50:131
1697699221,17cyclictest19811-21apt-get20:20:131
1697699221,17cyclictest19811-21apt-get20:20:121
1696199229,5cyclictest12-21ksoftirqd/020:37:150
1696199229,4cyclictest12-21ksoftirqd/021:05:250
1696199228,8cyclictest12-21ksoftirqd/022:27:100
1696199227,5cyclictest131rcu_preempt23:10:110
1696199223,12cyclictest630-21Async09:36:310
1696199221,6cyclictest19558-21cron20:20:010
1696199221,18cyclictest18549-21cut19:45:150
16961992210,4cyclictest12-21ksoftirqd/000:10:050
16961992210,3cyclictest12-21ksoftirqd/021:26:230
1699599212,12cyclictest0-21swapper/320:30:043
16995992116,2cyclictest0-21swapper/323:00:133
16995992115,3cyclictest0-21swapper/322:55:133
16995992112,3cyclictest0-21swapper/321:45:133
1697699213,5cyclictest21-21ksoftirqd/121:00:161
1697699212,5cyclictest21-21ksoftirqd/119:30:141
1696199219,4cyclictest12-21ksoftirqd/020:55:010
1696199218,4cyclictest12-21ksoftirqd/020:10:270
1696199213,7cyclictest12-21ksoftirqd/019:50:110
1696199212,6cyclictest12-21ksoftirqd/021:55:130
1696199212,6cyclictest12-21ksoftirqd/021:55:130
1696199211,5cyclictest12-21ksoftirqd/023:20:120
1696199211,3cyclictest0-21swapper/021:20:180
1696199211,15cyclictest13742-21ssh22:10:140
16961992110,4cyclictest12-21ksoftirqd/019:10:220
16995992016,2cyclictest0-21swapper/321:05:123
16995992014,4cyclictest35966irq/58-eth021:10:283
16995992014,4cyclictest35966irq/58-eth021:10:283
16995992014,2cyclictest0-21swapper/300:30:103
16995992012,3cyclictest0-21swapper/320:50:133
16995992010,7cyclictest0-21swapper/321:55:203
16995992010,7cyclictest0-21swapper/321:55:203
1697699203,5cyclictest21-21ksoftirqd/121:05:151
1697699203,4cyclictest21-21ksoftirqd/119:55:211
1697699202,5cyclictest21-21ksoftirqd/120:00:111
1696199207,4cyclictest12-21ksoftirqd/020:27:150
1696199207,4cyclictest12-21ksoftirqd/019:35:170
1696199203,13cyclictest0-21swapper/020:55:120
16961992014,4cyclictest0-21swapper/019:40:020
1699599199,7cyclictest0-21swapper/300:15:153
1699599198,7cyclictest0-21swapper/322:35:163
16995991914,3cyclictest0-21swapper/322:20:113
16995991913,3cyclictest0-21swapper/319:15:113
16995991913,2cyclictest0-21swapper/322:30:133
16995991913,2cyclictest0-21swapper/322:10:123
16995991911,3cyclictest0-21swapper/320:05:133
16995991910,6cyclictest0-21swapper/323:40:223
16995991910,6cyclictest0-21swapper/323:25:273
16995991910,6cyclictest0-21swapper/322:25:313
1697699193,4cyclictest21-21ksoftirqd/119:10:201
1696199193,5cyclictest12-21ksoftirqd/020:05:280
1699599189,6cyclictest0-21swapper/319:32:163
1699599189,6cyclictest0-21swapper/300:36:063
16995991811,3cyclictest0-21swapper/319:25:123
16995991811,2cyclictest0-21swapper/319:50:133
16995991810,6cyclictest0-21swapper/321:15:103
1696199182,14cyclictest25038-21kthreadcore19:15:210
1699599179,6cyclictest0-21swapper/320:10:273
1699599178,6cyclictest0-21swapper/321:00:153
1699599172,9cyclictest0-21swapper/320:15:123
16995991713,2cyclictest0-21swapper/320:00:113
16995991712,2cyclictest0-21swapper/319:55:113
1699599171,13cyclictest0-21swapper/319:20:123
1699599166,7cyclictest0-21swapper/319:10:223
16995991611,3cyclictest0-21swapper/320:40:113
16995991611,2cyclictest0-21swapper/320:20:123
16995991611,2cyclictest0-21swapper/320:20:123
1699599157,3cyclictest0-21swapper/320:45:113
16995991511,2cyclictest0-21swapper/320:35:123
1697699151,10cyclictest17048-21latency_hist19:10:011
1696199142,3cyclictest131rcu_preempt19:10:010
1699599117,2cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional