You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 09:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon May 11, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
61072174101,24sleep30-21swapper/319:07:373
4388217097,25sleep10-21swapper/119:05:081
6109214895,25sleep00-21swapper/019:07:380
62082143108,24sleep20-21swapper/219:08:572
250821410,1sleep00-21swapper/022:52:180
612621070,5sleep228-21ksoftirqd/222:06:442
1810321060,2sleep10-21swapper/123:04:261
325592720,9sleep334-21ksoftirqd/321:38:563
169762680,1sleep00-21swapper/021:26:490
319642660,4chrt0-21swapper/123:14:371
223922630,2sleep10-21swapper/123:55:151
288762590,1sleep00-21swapper/022:24:170
51332540,1sleep10-21swapper/100:30:181
144162520,1sleep20-21swapper/221:25:152
313042510,2sleep00-21swapper/022:01:140
219132500,1sleep00-21swapper/023:06:190
117352500,1sleep00-21swapper/019:20:180
271722490,1sleep10-21swapper/122:46:161
250012480,1sleep00-21swapper/000:20:260
236262450,1sleep10-21swapper/121:31:441
145102450,3sleep314509-21irqstats23:25:163
130632410,1sleep30-21swapper/300:11:583
178812320,2sleep10-21swapper/119:35:161
89492300,1chrt0-21swapper/322:33:193
108152300,3sleep00-21swapper/023:22:270
46112290,8sleep228-21ksoftirqd/222:54:242
26122290,1sleep30-21swapper/322:52:253
656699286,5cyclictest34-21ksoftirqd/300:31:573
287342280,2sleep00-21swapper/021:35:240
656699271,13cyclictest141rcu_preempt22:04:053
656499278,16cyclictest22-21ksoftirqd/121:40:191
6563992723,2cyclictest3051-21ssh22:05:100
656399270,25cyclictest636-21runrttasks21:55:580
134702270,2sleep30-21swapper/321:25:003
121802270,3sleep00-21swapper/000:35:190
656699261,2cyclictest0-21swapper/322:05:373
6564992624,1cyclictest0-21swapper/119:11:031
656499261,24cyclictest0-21swapper/122:04:161
656499260,2cyclictest2914-21ssh22:05:031
656399261,24cyclictest0-21swapper/019:11:080
273272260,1sleep20-21swapper/223:35:012
105272260,2sleep20-21swapper/221:45:372
6566992514,8cyclictest12353-21wc22:35:233
6566992514,7cyclictest52850irq/140-enp3s0-21:40:123
656699251,23cyclictest0-21swapper/319:11:033
656699250,22cyclictest16073-21ntp_states20:45:223
6565992523,1cyclictest0-21swapper/222:04:562
6565992518,5cyclictest19351-21ssh23:29:132
6565992512,9cyclictest19698-21apt-get19:40:122
656599251,23cyclictest0-21swapper/221:57:312
656599251,23cyclictest0-21swapper/219:11:092
6563992512,11cyclictest12-21ksoftirqd/021:20:230
169362250,2sleep10-21swapper/122:15:151
7492240,10sleep228-21ksoftirqd/200:26:552
7492240,10sleep228-21ksoftirqd/200:26:542
656699248,14cyclictest0-21swapper/323:50:163
6566992415,2cyclictest0-21swapper/323:01:173
6566992414,7cyclictest28595-21df20:00:143
656499247,15cyclictest22-21ksoftirqd/122:59:441
656499247,15cyclictest22-21ksoftirqd/122:59:441
6564992410,3cyclictest141rcu_preempt00:24:041
30832240,10sleep122-21ksoftirqd/123:16:261
131672240,2sleep20-21swapper/220:40:132
93862230,2sleep00-21swapper/022:57:280
93862230,2sleep00-21swapper/022:57:270
9042230,1sleep10-21swapper/122:50:421
656699234,11cyclictest0-21swapper/322:11:053
6566992314,7cyclictest30390-21df22:25:133
656699231,15cyclictest5614-21sh22:55:113
656699231,15cyclictest5614-21sh22:55:103
656699231,13cyclictest141rcu_preempt22:24:133
6565992314,6cyclictest20483-21seq21:30:072
6565992314,6cyclictest16097-21open_inodes20:45:222
6565992313,7cyclictest10200-21diskstats22:10:142
6565992311,9cyclictest22859-21sendmail22:20:002
656499239,4cyclictest22-21ksoftirqd/100:08:201
656499239,3cyclictest22-21ksoftirqd/121:53:091
656499236,4cyclictest22-21ksoftirqd/121:19:111
656499235,8cyclictest141rcu_preempt23:50:171
656499235,3cyclictest141rcu_preempt20:25:131
656499233,6cyclictest15561-21fschecks_time20:45:171
656499232,16cyclictest19340-21ssh23:05:131
656499232,14cyclictest22052-21ssh21:55:131
6564992316,5cyclictest22-21ksoftirqd/120:15:001
6564992313,6cyclictest19110-21sendmail_mailtr22:40:201
6564992310,3cyclictest22-21ksoftirqd/123:35:141
656399232,14cyclictest1208-21apt-get21:40:040
6563992316,5cyclictest5380-21/usr/sbin/munin20:20:210
6563992315,6cyclictest13199-21cut19:25:140
155182230,1sleep00-21swapper/023:25:240
656699227,12cyclictest15566-21expr21:50:143
656699222,1cyclictest141rcu_preempt00:25:003
6566992216,3cyclictest25615-21diskmemload22:45:143
6566992214,6cyclictest34-21ksoftirqd/320:10:003
656699220,2cyclictest141rcu_preempt22:20:003
656599228,3cyclictest271732sh22:46:162
656499229,3cyclictest22-21ksoftirqd/123:45:491
656499229,3cyclictest22-21ksoftirqd/123:40:141
656499229,3cyclictest22-21ksoftirqd/123:40:131
656499229,3cyclictest22-21ksoftirqd/123:31:421
656499229,3cyclictest22-21ksoftirqd/122:12:221
656499229,3cyclictest22-21ksoftirqd/121:46:401
656499229,3cyclictest22-21ksoftirqd/121:10:451
656499229,3cyclictest22-21ksoftirqd/100:00:131
656499228,2cyclictest141rcu_preempt23:20:231
656499228,12cyclictest22-21ksoftirqd/122:20:041
656499222,16cyclictest75-21kswapd000:25:131
656499222,16cyclictest75-21kswapd000:25:121
6564992213,5cyclictest52950irq/141-enp3s0-00:10:241
656499221,15cyclictest11060-21ssh00:35:041
6564992210,3cyclictest22-21ksoftirqd/121:36:021
6564992210,3cyclictest22-21ksoftirqd/120:55:161
6564992210,3cyclictest22-21ksoftirqd/119:45:251
6564992210,3cyclictest22-21ksoftirqd/100:18:411
6563992213,7cyclictest26413-21cat21:10:140
223472220,1sleep10-21swapper/121:00:181
186332220,2sleep30-21swapper/321:28:293
178592220,1sleep20-21swapper/223:04:122
73772210,1sleep30-21swapper/323:20:133
656699212,13cyclictest16532-21ssh00:15:123
6566992114,2cyclictest34-21ksoftirqd/321:31:043
656699211,2cyclictest141rcu_preempt23:14:093
656699211,2cyclictest141rcu_preempt00:27:193
656699211,2cyclictest141rcu_preempt00:27:183
656699210,8cyclictest31580-21runrttasks21:14:133
656599211,5cyclictest0-21swapper/221:50:532
6565992113,5cyclictest278-21systemd-journal00:00:012
6565992112,7cyclictest1905-21date21:15:262
6565992110,8cyclictest13651-21cat19:25:182
656499219,3cyclictest22-21ksoftirqd/123:25:101
656499219,3cyclictest22-21ksoftirqd/122:35:281
656499219,3cyclictest22-21ksoftirqd/122:30:191
656499219,3cyclictest22-21ksoftirqd/122:25:281
656499219,3cyclictest22-21ksoftirqd/121:28:191
656499219,3cyclictest22-21ksoftirqd/120:30:221
656499219,3cyclictest22-21ksoftirqd/120:05:131
656499218,3cyclictest22-21ksoftirqd/121:05:151
656499218,3cyclictest22-21ksoftirqd/119:55:131
656499217,3cyclictest141rcu_preempt21:20:131
6563992114,4cyclictest30655-21sh23:13:200
6563992112,7cyclictest4089-21cat22:30:000
6563992112,7cyclictest27769-21seq19:57:130
6563992111,7cyclictest560-21dbus-daemon21:35:010
54612210,2sleep00-21swapper/023:18:510
91542200,10sleep012-21ksoftirqd/021:45:150
656699208,3cyclictest34-21ksoftirqd/300:00:143
656699202,12cyclictest2271-21apt-get20:15:123
6566992016,2cyclictest10807-21ssh21:45:543
656699201,2cyclictest141rcu_preempt23:57:203
6566992012,6cyclictest1833-21sed23:15:263
6566992011,4cyclictest0-21swapper/300:35:043
656699201,11cyclictest141rcu_preempt00:05:493
6566992010,8cyclictest31081-21ssh23:37:163
656699200,2cyclictest141rcu_preempt23:31:173
656599206,3cyclictest256152diskmemload00:22:122
656599204,4cyclictest28-21ksoftirqd/200:30:142
6565992014,4cyclictest10542-21kernelversion00:10:172
6565992012,6cyclictest23231-21seq21:02:012
6565992012,6cyclictest22743-21ntpq19:45:202
6565992012,5cyclictest5957-21grep22:30:252
6565992010,7cyclictest20954-21unixbench_multi23:05:262
6565992010,3cyclictest167152seq00:15:142
656499209,3cyclictest22-21ksoftirqd/119:20:221
656499208,3cyclictest22-21ksoftirqd/120:40:111
6564992012,6cyclictest21524-21seq19:44:251
656399209,5cyclictest12611-21expr23:00:140
656399203,3cyclictest0-21swapper/023:40:130
656399203,3cyclictest0-21swapper/023:40:130
6563992013,4cyclictest13277-21sh22:35:530
6563992012,6cyclictest4035-21netstat00:05:200
6563992012,6cyclictest24661-21awk22:45:120
6563992012,6cyclictest21339-21expr23:30:160
6563992012,6cyclictest19892-21fw_packets20:55:140
6563992012,6cyclictest15712-21smartctl20:45:170
6563992012,5cyclictest25615-21diskmemload22:40:210
6563992011,7cyclictest16079-21ntpq19:30:200
6563992011,6cyclictest278-21systemd-journal20:45:010
6563992010,7cyclictest560-21dbus-daemon23:55:230
656399200,8cyclictest16916-21ssh22:15:130
114672200,9sleep228-21ksoftirqd/223:23:062
113752200,1sleep00-21swapper/023:46:310
656699197,3cyclictest34-21ksoftirqd/319:55:223
656699192,4cyclictest0-21swapper/323:49:113
656699192,2cyclictest141rcu_preempt22:42:143
656699192,11cyclictest19159-21ssh23:05:043
656699191,4cyclictest24190-21perl21:05:173
6566991912,5cyclictest17270-21apt-key19:35:003
6566991911,6cyclictest52850irq/140-enp3s0-21:15:143
656699191,16cyclictest34-21ksoftirqd/319:53:123
6566991911,6cyclictest13596-21grep20:40:163
6566991910,6cyclictest34-21ksoftirqd/320:30:123
656599197,7cyclictest17823-21seq22:40:092
656599192,9cyclictest15296-21ssh23:50:132
6565991913,4cyclictest29774-21sh21:36:082
6565991913,4cyclictest24268-21sh23:32:122
6565991913,4cyclictest14108-21sh23:49:162
6565991912,5cyclictest3551-21ls20:15:242
6565991912,5cyclictest16741-21seq19:30:532
6565991911,6cyclictest5915-21seq23:19:182
6565991911,6cyclictest5229-21seq23:42:062
6565991911,6cyclictest5229-21seq23:42:052
6565991911,6cyclictest24299-21hostname21:05:182
6565991911,6cyclictest17575-21wc00:40:002
6565991911,5cyclictest53050irq/142-enp3s0-21:40:132
6565991910,7cyclictest26436-21df19:55:142
6565991910,6cyclictest10833-21date19:20:012
656599190,16cyclictest4818-21cut20:20:142
656499198,3cyclictest22-21ksoftirqd/120:15:201
656499198,3cyclictest22-21ksoftirqd/120:00:121
656499197,2cyclictest22-21ksoftirqd/119:50:121
656499192,11cyclictest13051-21apt-get19:25:141
6564991912,5cyclictest5318-21ntp_kernel_pll_20:20:191
6564991911,6cyclictest16068-21ntp_kernel_err19:30:191
656399199,6cyclictest52750irq/139-enp3s0-00:25:120
656399199,6cyclictest52750irq/139-enp3s0-00:25:120
656399191,4cyclictest5294-21/usr/sbin/munin00:30:200
6563991912,5cyclictest32109-21seq20:07:440
6563991912,4cyclictest19710-21sh00:16:550
6563991912,4cyclictest16900-21unixbench_multi23:50:260
6563991911,6cyclictest9568-21date19:15:180
6563991911,6cyclictest29922-21seq20:02:080
6563991911,6cyclictest27625-21awk23:35:140
6563991911,6cyclictest24892-21seq19:50:200
6563991911,6cyclictest15953-21smartctl21:50:170
6563991911,6cyclictest11311-21cat00:10:240
6563991911,5cyclictest11371-21cat22:10:250
118362190,2chrt0-21swapper/222:35:182
110202190,2sleep20-21swapper/222:59:062
110202190,2sleep20-21swapper/222:59:052
656699189,6cyclictest6310-21cron20:24:593
656699182,2cyclictest141rcu_preempt23:41:283
656699182,2cyclictest141rcu_preempt23:41:273
656699182,13cyclictest17827-21cut20:50:153
656699181,7cyclictest141rcu_preempt21:58:383
6566991813,3cyclictest18191-21kernelversion19:35:183
6566991811,5cyclictest9987-21find19:15:233
6566991811,5cyclictest32755-21expr20:10:123
6566991811,5cyclictest13876-21ls19:25:203
6566991810,5cyclictest6914-21entropy20:25:153
6566991810,5cyclictest21990-21fschecks_time21:00:153
656599189,6cyclictest53050irq/142-enp3s0-00:05:132
656599183,12cyclictest32513-21wc21:15:002
656599181,8cyclictest32343-21idleruntime-cro23:15:012
6565991813,3cyclictest30974-21sh00:00:552
6565991813,3cyclictest21316-21sh21:00:002
6565991812,4cyclictest6950-21sh21:20:022
6565991812,4cyclictest32530-21sh22:26:112
6565991811,5cyclictest23937-21expr22:20:172
6565991811,5cyclictest18446-21sensors20:50:232
6565991811,5cyclictest11716-21cat19:20:172
6565991810,6cyclictest7484-21ntpq20:25:212
6565991810,6cyclictest24300-21df19:50:122
656499181,12cyclictest8748-21apt-get19:15:121
656399189,6cyclictest2874-21cat20:15:170
656399181,7cyclictest5100-21irqstats22:30:190
6563991813,3cyclictest32705-21sh21:15:070
6563991813,3cyclictest18003-21latency20:50:180
6563991811,5cyclictest20263-21hostname19:40:170
6563991811,5cyclictest19652-21cat19:40:000
6563991811,5cyclictest1885-21seq20:13:270
6563991811,5cyclictest10495-21seq20:33:490
6563991810,6cyclictest8151-21seq20:26:100
6563991810,5cyclictest25615-21diskmemload00:02:400
656699172,8cyclictest141rcu_preempt20:35:183
656599179,6cyclictest32743-21ls20:10:122
656599179,5cyclictest9690-21pmu-power20:30:212
6565991711,4cyclictest29339-21processes20:00:222
656499179,6cyclictest22-21ksoftirqd/120:50:131
6564991710,5cyclictest11411-21smartctl20:35:171
656399179,6cyclictest12306-21unixbench_singl20:35:250
656399178,6cyclictest21672-21grep21:00:130
6563991712,3cyclictest23705-21/usr/sbin/munin21:05:230
6563991710,5cyclictest22937-21readlink19:45:220
656699162,7cyclictest141rcu_preempt19:45:133
656699162,2cyclictest141rcu_preempt20:55:143
656699162,2cyclictest141rcu_preempt19:20:143
6565991611,3cyclictest17944-21sort19:35:162
6565991611,3cyclictest11518-21kernelversion20:35:182
656699152,2cyclictest141rcu_preempt19:40:123
6565991510,3cyclictest31515-21mailstats20:05:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional