You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-18 - 07:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Wed Mar 18, 2026 00:45:20)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1727399653,8cyclictest131rcu_preempt20:45:120
1728499565,45cyclictest29964-21ssh22:55:121
17273995324,19cyclictest12-21ksoftirqd/022:00:140
17273995324,19cyclictest12-21ksoftirqd/022:00:140
17273995118,23cyclictest12-21ksoftirqd/021:35:120
17273995018,6cyclictest12-21ksoftirqd/021:15:110
1727399491,8cyclictest131rcu_preempt19:10:120
1730599483,29cyclictest28366irq/58-eth023:45:143
1728499483,33cyclictest1200-21ssh21:45:121
1727399483,35cyclictest10229-21ssh23:20:140
17273994422,11cyclictest131rcu_preempt21:10:120
1730599431,24cyclictest28366irq/58-eth023:25:113
1728499422,30cyclictest24581-21ssh21:40:121
1730599414,25cyclictest28366irq/58-eth022:15:123
17294994126,7cyclictest27-21ksoftirqd/221:10:112
1729499404,8cyclictest131rcu_preempt00:25:112
1729499402,21cyclictest16645-21grep22:10:222
1729499392,16cyclictest261rcuc/219:40:122
1729499392,16cyclictest261rcuc/219:40:112
1728499391,28cyclictest20971-21ssh22:50:111
1728499385,22cyclictest21161-21ssh21:20:131
17294993730,3cyclictest9061-21kworker/u8:122:45:122
17294993730,3cyclictest9061-21kworker/u8:122:45:112
1729499372,19cyclictest26432-21ssh21:58:232
1728499379,13cyclictest0-21swapper/123:30:131
1730599362,24cyclictest28366irq/58-eth020:05:143
1730599362,24cyclictest28366irq/58-eth020:05:143
1728499362,30cyclictest4838-21ssh22:05:151
1728499333,21cyclictest22705-21ssh23:45:121
1727399331,25cyclictest26335-21ssh00:05:130
1729499322,25cyclictest261rcuc/220:30:142
1729499322,25cyclictest261rcuc/220:30:142
17284993223,4cyclictest15159-21apt-get20:15:141
1727399321,28cyclictest0-21swapper/021:55:120
1730599311,17cyclictest28366irq/58-eth023:55:123
1729499312,24cyclictest23609-21ssh23:09:232
1728499317,17cyclictest13736-21ssh23:40:121
1727399311,24cyclictest26369-21ssh22:35:130
1730599303,16cyclictest28366irq/58-eth022:45:123
1730599303,16cyclictest28366irq/58-eth022:45:113
1730599302,18cyclictest28366irq/58-eth021:40:113
1730599302,17cyclictest28366irq/58-eth023:00:143
1729499305,9cyclictest17405-21ssh22:30:132
1729499305,13cyclictest22346irq/40-dwc2_hso21:35:112
1729499303,22cyclictest0-21swapper/221:29:222
1729499300,22cyclictest22346irq/40-dwc2_hso19:10:122
1730599296,14cyclictest28366irq/58-eth000:05:123
1730599294,15cyclictest28366irq/58-eth000:10:123
1730599294,15cyclictest28366irq/58-eth000:10:113
1730599293,13cyclictest0-21swapper/323:09:223
17305992913,11cyclictest28366irq/58-eth023:35:133
17305992913,11cyclictest28366irq/58-eth023:35:123
17284992923,3cyclictest13829-21apt-get22:10:131
1728499291,8cyclictest21-21ksoftirqd/123:30:011
17284992912,7cyclictest21-21ksoftirqd/122:37:401
1727399293,17cyclictest8459-21ssh22:25:120
1730599283,15cyclictest0-21swapper/322:50:113
1729499283,20cyclictest0-21swapper/200:14:222
1729499283,20cyclictest0-21swapper/200:14:212
1729499281,11cyclictest4604-21grep20:00:292
17284992810,6cyclictest0-21swapper/100:20:141
1727399284,14cyclictest2889-21ssh00:10:120
1727399284,14cyclictest2889-21ssh00:10:120
1727399282,20cyclictest29175-21ssh21:24:230
1730599273,18cyclictest0-21swapper/321:30:123
1730599273,16cyclictest0-21swapper/321:24:233
1730599273,15cyclictest0-21swapper/322:14:233
1730599272,12cyclictest28366irq/58-eth022:00:143
1730599272,12cyclictest28366irq/58-eth022:00:143
1730599271,17cyclictest28366irq/58-eth023:20:143
1729499273,18cyclictest0-21swapper/221:24:222
1729499272,19cyclictest0-21swapper/219:45:122
17294992710,14cyclictest24317-21kthreadcore22:15:192
1728499272,19cyclictest26392-21cat22:15:231
1728499272,19cyclictest12923-21ssh23:02:221
1728499270,20cyclictest8256-21ssh23:55:111
1727399273,3cyclictest131rcu_preempt19:40:140
1727399273,3cyclictest131rcu_preempt19:40:130
1730599263,15cyclictest0-21swapper/321:58:233
1729499261,15cyclictest30745-21apt-get21:05:132
1728499266,5cyclictest21-21ksoftirqd/123:15:231
1728499263,14cyclictest2899-21diskmemload00:30:151
1728499262,17cyclictest31687-21ssh23:50:131
17284992618,4cyclictest29573-21apt-get20:30:141
17284992618,4cyclictest29573-21apt-get20:30:141
1727399266,8cyclictest131rcu_preempt22:30:140
1727399262,6cyclictest8374-21df23:55:140
1727399260,13cyclictest15509-21ntp_192.168.11422:45:230
1727399260,13cyclictest15509-21ntp_192.168.11422:45:220
1730599254,16cyclictest28366irq/58-eth019:10:113
1730599254,14cyclictest0-21swapper/321:45:143
1729499252,8cyclictest10371-21ssh22:06:212
1729499252,18cyclictest2232-21ssh22:02:232
1729499252,18cyclictest2232-21ssh22:02:222
1729499252,15cyclictest22346irq/40-dwc2_hso20:45:112
17294992512,8cyclictest0-21swapper/222:37:222
1729499251,11cyclictest12193-21ssh21:15:112
1728499257,6cyclictest21-21ksoftirqd/123:10:231
1728499257,5cyclictest21-21ksoftirqd/100:20:021
1728499253,6cyclictest21-21ksoftirqd/122:40:131
1728499253,4cyclictest21-21ksoftirqd/122:30:291
1728499253,4cyclictest21-21ksoftirqd/100:10:131
1728499253,4cyclictest21-21ksoftirqd/100:10:121
1728499252,10cyclictest15525-21ssh23:05:141
17284992517,3cyclictest3291-21apt-get21:10:141
1728499251,22cyclictest6648-21ssh21:30:121
17284992511,12cyclictest21-21ksoftirqd/100:35:111
1727399255,11cyclictest4754-21ssh23:35:130
1727399255,11cyclictest4754-21ssh23:35:120
1727399252,18cyclictest10372-21apt-get20:10:130
1730599246,7cyclictest0-21swapper/300:30:143
1730599242,15cyclictest0-21swapper/300:25:143
1729499247,5cyclictest0-21swapper/200:35:152
1729499245,3cyclictest131rcu_preempt19:15:152
1729499245,12cyclictest22346irq/40-dwc2_hso22:20:112
1729499243,16cyclictest0-21swapper/222:58:222
1729499241,6cyclictest1964866ssh00:00:222
17294992412,9cyclictest0-21swapper/221:30:302
1729499241,16cyclictest22414-21kthreadcore00:20:202
17294992411,6cyclictest22046irq/40-fe98000000:15:122
1728499247,5cyclictest21-21ksoftirqd/121:15:141
1728499247,4cyclictest0-21swapper/122:47:161
1728499247,4cyclictest0-21swapper/122:47:161
1728499246,4cyclictest21-21ksoftirqd/121:00:231
1728499245,5cyclictest21-21ksoftirqd/100:05:141
1728499244,7cyclictest21-21ksoftirqd/122:25:131
1728499244,4cyclictest21-21ksoftirqd/120:50:151
1728499243,5cyclictest21-21ksoftirqd/123:21:241
1728499243,5cyclictest21-21ksoftirqd/121:05:271
1728499242,17cyclictest31853-21ssh22:20:111
1727399245,10cyclictest0-21swapper/020:40:120
17273992411,3cyclictest12-21ksoftirqd/023:45:140
17273992410,7cyclictest0-21swapper/020:30:140
17273992410,7cyclictest0-21swapper/020:30:130
1730599233,13cyclictest0-21swapper/323:32:223
1730599230,13cyclictest28366irq/58-eth019:50:133
1729499232,2cyclictest131rcu_preempt21:44:582
1729499231,9cyclictest24683-21grep23:10:142
17294992317,4cyclictest261rcuc/223:35:142
17294992317,4cyclictest261rcuc/223:35:142
1729499231,3cyclictest0-21swapper/200:07:272
17294992312,9cyclictest27-21ksoftirqd/223:45:212
1728499235,6cyclictest21-21ksoftirqd/119:25:181
1728499235,6cyclictest21-21ksoftirqd/119:25:181
1728499235,11cyclictest21-21ksoftirqd/121:50:151
1728499234,5cyclictest21-21ksoftirqd/122:00:141
1728499234,5cyclictest21-21ksoftirqd/122:00:131
1728499233,4cyclictest21-21ksoftirqd/120:30:021
1728499232,4cyclictest21-21ksoftirqd/121:55:231
1728499231,3cyclictest8527-21ps23:35:291
1728499231,3cyclictest8527-21ps23:35:291
17284992313,7cyclictest0-21swapper/119:20:121
1727399231,8cyclictest8450-21ntp_192.168.11422:05:250
1730599227,10cyclictest28366irq/58-eth000:20:153
1730599223,12cyclictest0-21swapper/321:10:143
17305992211,9cyclictest0-21swapper/322:30:313
17305992211,9cyclictest0-21swapper/322:25:303
1729499221,4cyclictest0-21swapper/223:04:022
1729499221,3cyclictest0-21swapper/220:05:302
1729499221,3cyclictest0-21swapper/220:05:302
17294992211,9cyclictest11399-21ssh22:44:532
1729499221,15cyclictest24767-21find20:25:122
1729499220,3cyclictest0-21swapper/223:28:032
1728499226,10cyclictest21-21ksoftirqd/120:55:171
1728499226,10cyclictest21-21ksoftirqd/120:55:161
1728499224,4cyclictest21-21ksoftirqd/100:25:151
1728499222,15cyclictest15595-21apt-get21:35:101
1728499221,6cyclictest21-21ksoftirqd/119:45:261
1728499221,5cyclictest21-21ksoftirqd/121:25:131
17284992211,8cyclictest24539-21cron20:25:021
1728499221,11cyclictest21-21ksoftirqd/100:00:221
1727399222,17cyclictest935-21ls00:25:220
17273992212,7cyclictest683-21cron23:15:010
1727399221,18cyclictest26088-21cut21:00:160
17273992210,6cyclictest0-21swapper/022:16:160
17305992111,8cyclictest0-21swapper/300:04:563
17305992111,7cyclictest28366irq/58-eth022:35:133
1729499217,12cyclictest27-21ksoftirqd/200:30:242
1729499216,10cyclictest27-21ksoftirqd/220:15:132
1729499215,7cyclictest12683-21rm23:55:302
1729499214,13cyclictest22346irq/40-dwc2_hso23:20:142
1729499212,2cyclictest131rcu_preempt23:40:112
1729499211,5cyclictest0-21swapper/219:26:022
1729499211,5cyclictest0-21swapper/219:26:012
1729499211,14cyclictest6741-21find20:40:102
1729499211,11cyclictest536466ssh23:51:582
17294992110,4cyclictest8415-21/usr/sbin/munin22:25:142
1729499210,3cyclictest0-21swapper/222:54:442
1728499213,5cyclictest21-21ksoftirqd/119:50:121
1728499213,10cyclictest0-21swapper/119:55:121
1728499211,6cyclictest21-21ksoftirqd/120:50:021
17284992115,3cyclictest5566-21apt-get20:05:131
17284992115,3cyclictest5566-21apt-get20:05:121
17284992113,4cyclictest17505-21apt-get19:10:101
1728499211,14cyclictest6881-21cut20:40:131
1727399211,6cyclictest30691-21irqstats22:55:180
17273992113,5cyclictest0-21swapper/023:50:170
17273992112,7cyclictest21110-21sshd23:06:210
1727399211,15cyclictest19245-21switchtime00:35:290
1730599203,11cyclictest0-21swapper/322:05:223
1730599202,12cyclictest28366irq/58-eth023:10:133
1730599202,10cyclictest28366irq/58-eth022:20:113
17305992013,4cyclictest0-21swapper/300:35:233
1730599201,15cyclictest28366irq/58-eth022:40:133
1729499209,9cyclictest13826-21kworker/u8:023:30:192
1729499207,10cyclictest1900-21irqrtprio23:15:202
1729499203,4cyclictest27-21ksoftirqd/221:50:262
1729499202,9cyclictest30335-21cut19:55:202
1729499201,9cyclictest1770-21irqrtprio21:45:182
1729499201,13cyclictest24976-21kthreadcore19:50:202
17294992010,8cyclictest0-21swapper/221:00:172
17294992010,8cyclictest0-21swapper/219:35:162
1729499200,11cyclictest2049-21fschecks_count20:35:172
1728499204,5cyclictest21-21ksoftirqd/120:35:241
1728499203,4cyclictest21-21ksoftirqd/120:05:021
1728499202,12cyclictest0-21swapper/119:30:111
1727399207,4cyclictest131rcu_preempt22:10:210
1727399207,3cyclictest2899-21diskmemload00:02:560
1727399204,14cyclictest1388-21uname21:25:250
1730599198,8cyclictest0-21swapper/323:16:503
1730599192,6cyclictest0-21swapper/322:55:123
1730599192,11cyclictest0-21swapper/319:30:123
1730599192,11cyclictest0-21swapper/319:25:143
1730599192,11cyclictest0-21swapper/319:25:133
1730599192,10cyclictest0-21swapper/319:20:123
17305991911,6cyclictest0-21swapper/321:50:323
1730599191,13cyclictest0-21swapper/320:30:173
1730599191,13cyclictest0-21swapper/320:30:163
17305991911,3cyclictest0-21swapper/320:10:123
1729499196,5cyclictest21031-21kthreadcore20:20:192
1729499192,14cyclictest12042-21kthreadcore20:10:202
1729499191,9cyclictest30908-21tail19:20:272
1729499191,9cyclictest24641-21wc20:55:272
1729499191,9cyclictest24641-21wc20:55:262
1729499191,15cyclictest16995-21kernelversion20:50:182
1729499191,10cyclictest4500-21wc19:30:152
1728499192,4cyclictest21-21ksoftirqd/119:40:271
1728499192,4cyclictest21-21ksoftirqd/119:40:271
1728499192,4cyclictest21-21ksoftirqd/119:15:151
1728499190,7cyclictest9249-21fschecks_time19:35:151
1727399198,3cyclictest131rcu_preempt20:20:280
1727399197,3cyclictest0-21swapper/021:45:040
1727399196,9cyclictest0-21swapper/020:05:130
1727399196,9cyclictest0-21swapper/020:05:120
1727399191,9cyclictest7370-21kernelversion21:30:180
1727399191,9cyclictest1343-21ssh22:20:190
17273991912,4cyclictest0-21swapper/021:45:010
17273991912,3cyclictest6445-21sh00:30:150
1727399191,12cyclictest8580-21grep23:00:230
1727399191,11cyclictest22412-21grep19:45:260
1730599188,8cyclictest0-21swapper/321:39:103
1730599183,11cyclictest0-21swapper/323:43:223
1730599183,11cyclictest0-21swapper/300:18:223
1730599183,10cyclictest0-21swapper/321:28:223
1730599182,10cyclictest0-21swapper/320:25:153
1730599182,10cyclictest0-21swapper/319:35:123
1730599182,10cyclictest0-21swapper/319:15:143
1730599181,12cyclictest0-21swapper/321:15:143
17305991810,5cyclictest28366irq/58-eth023:50:123
1729499181,9cyclictest17324-21latency_hist19:10:022
1728499182,4cyclictest21-21ksoftirqd/120:10:131
1727399189,7cyclictest30947-21perl21:05:170
1727399189,6cyclictest19848-21cut23:25:170
1727399183,12cyclictest30603-21ssh23:30:220
1727399182,13cyclictest9828-21kthreadcore19:35:180
1727399181,7cyclictest131rcu_preempt21:53:380
17273991811,5cyclictest0-21swapper/000:16:320
1727399181,13cyclictest21090-21df_inode22:50:130
1727399181,12cyclictest6612-21ssh22:40:260
1730599172,10cyclictest0-21swapper/321:00:153
1730599172,10cyclictest0-21swapper/320:45:133
1730599172,10cyclictest0-21swapper/320:00:133
1730599172,10cyclictest0-21swapper/319:40:053
1730599172,10cyclictest0-21swapper/319:40:043
1727399178,7cyclictest31696-21memory19:55:230
1727399174,11cyclictest23631-21kthreadcore19:15:190
1727399172,10cyclictest0-21swapper/020:15:140
1727399172,10cyclictest0-21swapper/020:00:130
1727399171,14cyclictest24468-21sort00:20:250
1727399171,13cyclictest27084-21apt-get19:20:120
1727399171,13cyclictest2570-21ssh23:15:200
1727399170,9cyclictest2763-21kthreadcore20:35:190
1727399170,14cyclictest14330-21irqrtprio23:40:170
1730599162,8cyclictest28366irq/58-eth020:40:103
1730599161,12cyclictest0-21swapper/319:55:283
1730599161,10cyclictest0-21swapper/321:10:013
1728499161,12cyclictest17345-21apt-key19:10:021
1727399164,10cyclictest20308-21usage_Anybus20:50:290
1727399163,10cyclictest3300-21unixbench_multi19:25:280
1727399163,10cyclictest3300-21unixbench_multi19:25:280
1727399162,4cyclictest131rcu_preempt20:55:220
1727399162,4cyclictest131rcu_preempt20:55:210
1727399162,11cyclictest25146-21kthreadcore19:50:190
1727399161,4cyclictest0-21swapper/019:35:010
1727399161,12cyclictest27461-21tr20:25:220
1730599152,10cyclictest0-21swapper/320:20:123
1730599151,8cyclictest0-21swapper/320:35:153
1727399153,10cyclictest17266-21cyclictest19:10:010
1730599149,3cyclictest0-21swapper/319:48:163
1730599138,3cyclictest0-21swapper/320:53:103
1730599131,9cyclictest0-21swapper/320:55:153
1730599131,9cyclictest0-21swapper/320:55:153
1730599131,10cyclictest0-21swapper/320:15:173
1730599122,7cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional