You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 05:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Fri Sep 18, 2026 00:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
101612176103,28sleep10-21swapper/119:06:241
102992173103,27sleep30-21swapper/319:08:103
103932171119,29sleep00-21swapper/019:09:240
102632162105,28sleep20-21swapper/219:07:422
226502820,2sleep00-21swapper/019:35:230
199532750,1sleep20-21swapper/222:15:012
238552700,1sleep00-21swapper/021:53:400
107002630,1sleep30-21swapper/300:08:583
42462570,8sleep228-21ksoftirqd/200:04:112
46972560,2sleep20-21swapper/221:39:372
286902550,3sleep028683-21apt-get23:09:590
157732530,2sleep10-21swapper/123:48:171
38302510,2sleep20-21swapper/222:02:372
35342510,2sleep10-21swapper/121:14:281
231962490,3sleep223176-21munin-run23:30:002
184462490,3sleep30-21swapper/322:37:403
315132470,2sleep10-21swapper/122:23:211
176212470,1sleep00-21swapper/019:25:160
93962460,1chrt0-21swapper/121:18:361
281522390,2sleep128153-21diskmemload21:56:131
249372380,3sleep00-21swapper/022:18:310
167202380,2chrt0-21swapper/200:13:192
115522350,1sleep30-21swapper/300:33:503
117182310,12sleep334-21ksoftirqd/321:20:153
10715992922,4cyclictest17537-21diskstats19:25:152
1071499297,7cyclictest22-21ksoftirqd/123:30:151
10714992925,3cyclictest211rcuc/121:20:551
1071499291,26cyclictest27381-21seq19:46:341
1071499291,25cyclictest15109-21/usr/sbin/munin19:20:121
1071499286,8cyclictest22-21ksoftirqd/100:20:141
1071499286,10cyclictest22-21ksoftirqd/122:05:091
1071499284,2cyclictest141rcu_preempt23:00:001
10714992823,3cyclictest211rcuc/100:26:021
10714992816,5cyclictest22-21ksoftirqd/100:00:001
1071499281,25cyclictest0-21swapper/119:32:021
10714992810,9cyclictest22-21ksoftirqd/121:47:381
1071399281,3cyclictest25311-21chrt19:42:250
1071399281,25cyclictest0-21swapper/020:34:500
1071399281,25cyclictest0-21swapper/020:18:570
1071699272,5cyclictest34-21ksoftirqd/321:40:283
1071499279,10cyclictest22-21ksoftirqd/122:50:351
1071499278,9cyclictest22-21ksoftirqd/100:30:371
1071499278,8cyclictest22-21ksoftirqd/122:49:581
1071499277,9cyclictest22-21ksoftirqd/122:11:161
1071499277,11cyclictest22-21ksoftirqd/123:26:421
1071499276,9cyclictest22-21ksoftirqd/123:41:411
1071499275,12cyclictest211rcuc/121:30:151
1071499272,4cyclictest22-21ksoftirqd/119:20:011
10714992723,3cyclictest211rcuc/122:29:271
10714992723,3cyclictest211rcuc/121:50:431
1071499271,24cyclictest0-21swapper/120:18:481
1071499271,24cyclictest0-21swapper/119:55:141
1071499271,24cyclictest0-21swapper/119:52:381
1071499271,24cyclictest0-21swapper/119:42:251
1071499271,24cyclictest0-21swapper/119:36:251
1071499271,24cyclictest0-21swapper/119:25:241
1071499271,1cyclictest0-21swapper/120:11:041
1071399271,24cyclictest0-21swapper/020:41:310
1071399271,24cyclictest0-21swapper/019:55:120
1071399271,24cyclictest0-21swapper/019:32:450
91202260,2sleep30-21swapper/323:19:183
239852260,5sleep3341ksoftirqd/321:53:473
17912260,1chrt0-21swapper/300:25:443
1071699261,6cyclictest13338-21cstates00:35:143
1071699261,2cyclictest0-21swapper/319:33:283
1071699261,23cyclictest0-21swapper/320:35:093
1071699261,23cyclictest0-21swapper/320:16:363
1071699261,23cyclictest0-21swapper/320:12:173
1071699261,23cyclictest0-21swapper/319:27:083
1071699261,23cyclictest0-21swapper/319:21:163
1071599261,24cyclictest0-21swapper/219:47:262
1071599261,23cyclictest0-21swapper/220:35:102
1071599261,23cyclictest0-21swapper/220:33:292
1071599261,23cyclictest0-21swapper/220:27:472
1071499269,9cyclictest22-21ksoftirqd/122:01:571
1071499269,9cyclictest22-21ksoftirqd/121:40:331
1071499269,8cyclictest22-21ksoftirqd/122:34:551
1071499268,9cyclictest22-21ksoftirqd/123:51:041
1071499267,10cyclictest22-21ksoftirqd/100:13:371
1071499266,10cyclictest22-21ksoftirqd/100:03:171
1071499265,12cyclictest22-21ksoftirqd/123:22:531
1071499265,10cyclictest22-21ksoftirqd/123:07:431
10714992624,1cyclictest0-21swapper/121:01:261
1071499261,9cyclictest22-21ksoftirqd/120:30:161
10714992619,5cyclictest22-21ksoftirqd/100:20:011
1071499261,24cyclictest0-21swapper/120:41:181
1071499261,24cyclictest0-21swapper/120:36:311
1071499261,24cyclictest0-21swapper/120:20:311
1071499261,1cyclictest0-21swapper/120:27:411
10714992611,9cyclictest22-21ksoftirqd/123:15:001
10713992624,1cyclictest0-21swapper/020:37:450
10713992624,1cyclictest0-21swapper/020:20:590
1071399261,24cyclictest0-21swapper/020:09:400
1071399261,23cyclictest0-21swapper/020:10:470
1071399260,24cyclictest0-21swapper/019:45:020
1071399260,24cyclictest0-21swapper/019:20:010
1071699258,9cyclictest34-21ksoftirqd/300:11:433
10716992523,1cyclictest0-21swapper/319:38:223
10716992522,2cyclictest0-21swapper/319:54:453
10716992522,2cyclictest0-21swapper/319:41:123
10716992522,1cyclictest0-21swapper/319:55:103
1071699251,23cyclictest0-21swapper/320:41:253
1071699251,23cyclictest0-21swapper/320:34:123
1071699251,1cyclictest0-21swapper/321:01:543
1071699251,18cyclictest0-21swapper/319:45:103
10715992523,1cyclictest0-21swapper/220:42:102
10715992523,1cyclictest0-21swapper/219:54:352
10715992522,2cyclictest0-21swapper/220:18:422
1071599251,23cyclictest0-21swapper/220:49:042
1071599251,23cyclictest0-21swapper/219:39:092
1071599251,23cyclictest0-21swapper/219:30:062
1071599251,1cyclictest0-21swapper/220:11:322
1071599250,24cyclictest0-21swapper/219:41:262
1071599250,23cyclictest0-21swapper/220:20:372
1071599250,23cyclictest0-21swapper/219:55:082
1071499258,9cyclictest22-21ksoftirqd/100:38:441
1071499257,9cyclictest22-21ksoftirqd/122:36:371
1071499257,10cyclictest22-21ksoftirqd/100:07:341
10714992522,2cyclictest211rcuc/122:40:461
10714992522,2cyclictest211rcuc/121:35:281
10714992520,3cyclictest211rcuc/123:04:341
10714992519,4cyclictest22-21ksoftirqd/123:20:011
1071499251,1cyclictest0-21swapper/120:04:041
10713992523,1cyclictest0-21swapper/020:28:220
10713992523,1cyclictest0-21swapper/020:04:130
10713992523,1cyclictest0-21swapper/019:52:460
10713992517,6cyclictest28525-21ssh21:56:350
1071399251,1cyclictest0-21swapper/021:01:540
1071399250,5cyclictest1118-21awk22:00:230
41522240,2sleep34117-21munin-run23:40:013
292212240,1sleep30-21swapper/322:21:033
229262240,7sleep012-21ksoftirqd/023:29:470
10716992422,1cyclictest0-21swapper/320:28:523
10716992422,1cyclictest0-21swapper/320:21:193
10716992422,1cyclictest0-21swapper/320:00:593
10716992421,2cyclictest0-21swapper/320:09:473
1071699241,6cyclictest0-21swapper/300:20:143
10716992414,6cyclictest10559-21expr23:20:133
1071499244,9cyclictest22-21ksoftirqd/121:25:011
1071499243,7cyclictest22-21ksoftirqd/123:35:161
1071499242,5cyclictest22-21ksoftirqd/122:15:141
1071399243,16cyclictest24546-21ssh21:30:100
10713992416,6cyclictest5681-21ssh00:29:410
10713992411,10cyclictest5068-21grep23:15:240
183952230,2sleep30-21swapper/321:49:523
1071599238,3cyclictest292972sh22:21:092
10715992314,6cyclictest15627-21ssh21:47:062
10715992312,4cyclictest28-21ksoftirqd/222:30:002
1071499233,5cyclictest22-21ksoftirqd/120:50:001
10713992321,1cyclictest0-21swapper/019:19:250
10713992317,4cyclictest12-21ksoftirqd/021:30:000
48072220,2sleep00-21swapper/021:39:420
243372220,1sleep30-21swapper/322:17:543
125892221,18sleep212606-21sh00:34:552
1071699221,8cyclictest0-21swapper/300:15:203
10716992212,8cyclictest24739-21cat23:55:133
1071599229,7cyclictest19859-21rm21:25:422
1071599222,7cyclictest17142-21kworker/u8:221:15:162
10715992220,1cyclictest0-21swapper/221:01:552
10715992220,1cyclictest0-21swapper/220:02:492
10715992220,1cyclictest0-21swapper/219:20:102
1071599221,6cyclictest5215-21sed23:15:242
10715992214,6cyclictest11776-21ls23:20:252
1071499225,9cyclictest22-21ksoftirqd/120:06:201
1071399222,13cyclictest29769-21diskmemload22:05:030
1071399221,5cyclictest0-21swapper/000:10:000
10713992213,7cyclictest12016-21seq22:32:550
1071399220,10cyclictest66050lldpd23:03:440
222192210,1sleep20-21swapper/223:53:032
1071699213,13cyclictest31266-21ssh21:35:113
1071699211,6cyclictest0-21swapper/300:00:283
10716992113,6cyclictest18973-21ntp_states23:50:233
10716992113,5cyclictest0-21swapper/321:25:113
1071699211,16cyclictest4520-21ssh21:15:123
10716992110,6cyclictest53050irq/142-enp3s0-23:10:113
10716992110,6cyclictest53050irq/142-enp3s0-22:40:153
1071599214,4cyclictest32611-21fschecks_count00:25:152
1071599211,6cyclictest7346-21tune2fs22:05:152
10715992113,6cyclictest19388-21munin-run00:40:002
1071599211,2cyclictest0-21swapper/223:59:452
10715992112,6cyclictest24900-21df21:30:122
10715992112,3cyclictest22901-21grep23:05:142
10715992110,6cyclictest28-21ksoftirqd/221:53:512
10715992110,2cyclictest29769-21diskmemload22:56:212
1071499215,8cyclictest22-21ksoftirqd/120:57:351
1071499211,3cyclictest22-21ksoftirqd/119:10:221
1071399216,3cyclictest111782sh23:45:110
10713992115,4cyclictest12-21ksoftirqd/022:59:590
10713992113,6cyclictest4334-21seq22:50:540
10713992113,6cyclictest21875-21tune2fs20:50:160
10713992112,6cyclictest21347-21grep22:40:140
93122200,1sleep30-21swapper/323:43:303
41252200,1sleep20-21swapper/221:15:002
1071699209,8cyclictest23582-21apt-get20:55:123
1071699209,7cyclictest53050irq/142-enp3s0-23:25:243
1071699203,12cyclictest22251-21ssh23:05:043
1071699203,12cyclictest12900-21apt-get19:15:123
1071699202,13cyclictest6689-21sh22:05:043
10716992012,5cyclictest8722-21sh22:30:213
1071699200,6cyclictest29269-21cat22:45:243
1071599204,6cyclictest28-21ksoftirqd/222:34:372
1071599202,6cyclictest20237-21apt-get22:15:132
1071599201,7cyclictest5563-21apt-get21:40:132
10715992011,7cyclictest21783-21diskstats20:50:152
10715992010,7cyclictest11603-21df_inode21:20:142
10715992010,5cyclictest52850irq/140-enp3s0-23:40:122
1071499202,6cyclictest22-21ksoftirqd/120:50:261
1071399203,12cyclictest75-21kswapd000:15:110
10713992012,6cyclictest6033-21seq22:28:390
10713992012,6cyclictest21120-21seq20:49:140
10713992012,6cyclictest10731-21find19:10:110
10713992012,5cyclictest27655-21latency22:20:180
10713992011,7cyclictest13756-21cat00:10:280
10713992011,6cyclictest25680-21cpuspeed_turbos00:20:140
10713992010,6cyclictest11143-21load23:20:180
1071699192,5cyclictest26883-21ssh21:31:023
1071699192,11cyclictest30201-21apt-get21:10:123
10716991911,6cyclictest21836-21cut23:04:593
10716991911,6cyclictest1027-21ntp_states22:00:223
10716991911,4cyclictest13903-21sh22:10:133
10716991910,6cyclictest11595-21grep19:10:193
1071699190,16cyclictest25654-21taskset21:55:123
10715991913,3cyclictest26468-21sh22:44:012
10715991912,5cyclictest3305-21cat23:15:012
10715991912,5cyclictest23578-21/usr/sbin/munin23:30:222
10715991912,4cyclictest32712-21sh23:36:092
10715991911,6cyclictest5895-21entropy00:05:162
10715991911,6cyclictest26566-21ntpq00:20:222
10715991911,6cyclictest14751-21cat22:35:162
10715991911,6cyclictest12138-21date23:45:192
10715991911,6cyclictest11334-21idleruntime19:10:152
10715991910,7cyclictest26094-21cut21:55:152
10715991910,7cyclictest21803-21basename23:05:002
1071499190,7cyclictest28704-21seq21:05:191
1071399191,7cyclictest4920-21df23:40:140
10713991912,4cyclictest30963-21sh23:10:260
10713991911,6cyclictest7294-21cat00:30:170
10713991911,6cyclictest25295-21seq20:57:570
10713991911,6cyclictest23610-21apt-get23:30:120
10713991910,7cyclictest6092-21fw_packets21:40:180
10713991910,6cyclictest13133-21apt-get00:35:120
1071699186,3cyclictest141rcu_preempt20:49:593
1071699183,11cyclictest8818-21apt-get22:55:133
1071699181,5cyclictest0-21swapper/322:25:403
1071699181,4cyclictest0-21swapper/323:32:063
10716991811,5cyclictest2976-21sed22:50:203
10716991811,5cyclictest28762-21ntpq21:05:213
1071699181,15cyclictest15627-21ssh23:48:093
1071599182,11cyclictest27889-21sh22:45:142
1071599181,3cyclictest0-21swapper/200:15:152
10715991813,3cyclictest18294-21munin-node20:55:102
1071599181,2cyclictest0-21swapper/219:15:252
10715991812,4cyclictest2723-21missed_timers20:05:212
1071599181,11cyclictest2377-21cstates22:50:142
1071399189,7cyclictest19510-21sed23:50:280
1071399189,7cyclictest11448-21expr21:20:130
1071399181,6cyclictest0-21swapper/022:10:160
10713991813,3cyclictest32486-21proc_pri00:00:270
10713991813,3cyclictest16358-21sh21:47:500
10713991812,4cyclictest8277-21sh21:17:290
10713991812,4cyclictest27573-21sh23:56:440
10713991812,3cyclictest27708-21sh22:45:030
10713991811,5cyclictest14796-21expr22:35:160
10713991810,6cyclictest31049-21cat21:10:180
10713991810,5cyclictest52750irq/139-enp3s0-23:35:190
10716991710,5cyclictest21924-21grep20:50:153
10715991711,4cyclictest636-21runrttasks21:05:032
1071399169,5cyclictest28385-21gltestperf21:05:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional