You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 17:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu May 14, 2026 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6842221096,94sleep20-21swapper/207:05:072
85192176100,25sleep10-21swapper/107:07:051
6845217196,22sleep00-21swapper/007:05:090
84592140103,26sleep30-21swapper/307:06:203
220162610,0sleep0131rcuc/011:05:130
126202610,2sleep00-21swapper/011:45:500
127052600,1sleep30-21swapper/308:30:183
114242560,1sleep10-21swapper/112:10:071
124032550,3sleep30-21swapper/310:57:453
314682540,3sleep331462-21grep09:35:183
308852530,2sleep20-21swapper/209:35:132
294902510,1sleep00-21swapper/011:35:100
160562490,1sleep00-21swapper/011:25:070
84412480,1sleep20-21swapper/211:43:232
244602450,1sleep30-21swapper/311:30:243
52542430,2chrt0-21swapper/210:04:432
29972430,2sleep10-21swapper/112:27:201
232942410,2chrt0-21swapper/309:53:443
284202400,1sleep10-21swapper/109:33:201
276402380,2sleep20-21swapper/211:09:362
9016993712,6cyclictest5039-21cat09:15:240
9016993414,4cyclictest4928-21sh11:15:540
9016993412,6cyclictest12709-21vmstat07:15:260
9016993411,6cyclictest8386-21sed08:20:180
9016993411,5cyclictest11021-21cut07:15:000
9016993313,3cyclictest12156-21sh10:33:350
9016993311,6cyclictest12-21ksoftirqd/012:10:130
9016993310,7cyclictest829-21cut08:05:000
9016993215,3cyclictest15032-21ntp_offset10:35:200
9016993110,4cyclictest22832-21cpu11:30:110
901699302,11cyclictest26309-21apt-get10:20:120
9016993013,3cyclictest16478-21sh09:48:370
901699300,16cyclictest13503-21irqstats12:35:180
9016992913,3cyclictest14407-21sh10:10:260
9016992811,4cyclictest28795-21diskmemload12:16:000
61442270,2chrt0-21swapper/310:53:123
9018992623,2cyclictest0-21swapper/207:12:352
901799261,24cyclictest0-21swapper/107:12:381
309952260,3sleep330964-21munin-run07:59:593
104062260,1sleep30-21swapper/310:08:093
901999258,10cyclictest52850irq/140-enp3s0-11:00:123
9019992519,4cyclictest34-21ksoftirqd/307:20:003
901999251,23cyclictest0-21swapper/307:11:333
9019992511,8cyclictest9446-21ssh11:20:113
901799257,11cyclictest53050irq/142-enp3s0-12:00:121
9017992510,12cyclictest20558-21seq10:40:041
9016992517,6cyclictest6247-21cstates10:05:110
320862250,2sleep00-21swapper/011:12:190
901799243,14cyclictest22460-21chrt07:40:121
901799242,15cyclictest7314-21ssh10:30:121
9016992412,10cyclictest636-21runrttasks09:55:420
134112240,9sleep122-21ksoftirqd/109:45:321
901899230,20cyclictest20594-21ntpq10:15:192
901999222,14cyclictest19590-21ssh10:15:123
901999222,13cyclictest20700-21chrt10:40:113
9019992211,7cyclictest5860-21ntpq12:05:193
9019992210,10cyclictest2761-21cpu11:15:123
9018992214,6cyclictest23843-21ls08:55:252
9018992213,6cyclictest28904-21cat09:10:012
9018992213,6cyclictest18054-21apt-get07:30:092
9017992216,4cyclictest4402-21dump-pmu-power09:40:001
9017992212,7cyclictest18377-21df07:30:131
9017992210,8cyclictest6105-21grep09:40:231
9016992213,7cyclictest25899-21ls07:45:220
9016992213,6cyclictest19921-21cat10:15:130
27762220,2sleep00-21swapper/012:27:070
901999218,9cyclictest11505-21awk11:45:203
9019992115,4cyclictest34-21ksoftirqd/308:55:003
9019992113,6cyclictest2295-21cron11:15:003
9019992111,7cyclictest22935-21df_inode08:55:143
901899214,4cyclictest0-21swapper/209:59:262
9018992115,4cyclictest28-21ksoftirqd/208:20:002
9018992114,5cyclictest24340-21turbostat.cron07:44:592
9018992112,7cyclictest560-21dbus-daemon09:30:002
9018992111,7cyclictest25106-21python309:30:222
901799212,10cyclictest6036-21ssh12:30:101
901799212,10cyclictest6036-21ssh12:30:101
9017992114,4cyclictest53050irq/142-enp3s0-11:05:101
9017992112,7cyclictest28204-21seq12:21:491
9017992111,7cyclictest560-21dbus-daemon08:00:001
9017992111,7cyclictest16661-21fw_packets11:25:151
901699216,13cyclictest25342-21cat09:30:220
9016992115,4cyclictest31823-21grep10:00:110
9016992112,7cyclictest2681-21seq08:08:040
9016992112,6cyclictest3756-21cut08:10:140
298252210,2sleep20-21swapper/210:46:082
103742210,2sleep00-21swapper/009:44:140
96132200,2chrt0-21swapper/310:31:003
901999201,9cyclictest16261-21cut11:25:123
9019992014,4cyclictest34-21ksoftirqd/312:40:003
9019992014,4cyclictest34-21ksoftirqd/312:00:003
9019992014,4cyclictest18950irq/143-ahci[0012:20:003
9019992014,3cyclictest132882chrt09:45:263
9019992013,5cyclictest16652-21dump-pmu-power09:25:003
9019992011,7cyclictest13012-21cat12:10:243
901899202,13cyclictest22818-21ssh11:30:112
901899202,12cyclictest9428-21apt-get11:20:112
901899201,7cyclictest1483950ssh09:46:592
9018992015,3cyclictest1422-21latency10:25:192
9018992012,6cyclictest278-21systemd-journal12:00:002
9018992012,6cyclictest24829-21expr07:45:112
9018992012,6cyclictest16757-21hddtemp_smartct11:25:152
9018992012,6cyclictest13543-21cut10:10:182
9018992011,7cyclictest4121-21cat08:10:172
9018992011,6cyclictest4648-21open_files09:15:202
9018992011,6cyclictest16978-21cat11:50:002
901799209,3cyclictest31822-21sh10:24:141
901799202,12cyclictest11203-21apt-get07:15:111
901799201,9cyclictest4842-21ssh12:05:111
9017992015,3cyclictest7938-21cron10:55:001
9017992013,5cyclictest9134-21sed11:20:011
9017992013,5cyclictest22572-21seq08:55:061
9017992012,6cyclictest24505-21awk11:30:241
9017992012,5cyclictest26049-21grep07:45:231
9017992011,7cyclictest6799-21memory10:05:161
9017992010,6cyclictest14046-21mailstats12:35:231
9016992014,4cyclictest6969-21sh11:41:530
9016992014,4cyclictest21776-21sh09:52:120
9016992013,5cyclictest5462-21smartctl12:05:150
9016992013,5cyclictest19662-21seq07:31:360
9016992013,5cyclictest18075-21latency_hist08:45:000
9016992012,6cyclictest9503-21pmu-power10:55:220
9016992012,6cyclictest4945-21seq10:28:040
9016992012,6cyclictest31436-21ls09:35:180
9016992012,6cyclictest28484-21seq07:50:460
9016992012,6cyclictest26496-21seq09:03:130
80092200,1sleep30-21swapper/312:30:393
80092200,1sleep30-21swapper/312:30:393
901999191,6cyclictest636-21runrttasks10:37:363
901999191,4cyclictest0-21swapper/307:24:283
9019991913,4cyclictest14711-21sh10:10:433
9019991911,6cyclictest24794-21ntp_kernel_err09:30:193
9019991911,6cyclictest22397-21cat07:40:003
9019991911,6cyclictest1211-21expr08:05:123
9019991910,3cyclictest29854-21sh11:35:133
901899192,12cyclictest18397-21ssh12:15:112
901899192,12cyclictest17189-21apt-get11:50:122
901899192,11cyclictest26939-21apt-get07:50:102
9018991914,3cyclictest6977-21sh09:40:492
9018991914,3cyclictest6081-21sendmail_mailqu12:05:222
9018991913,4cyclictest4092-21sh12:28:282
9018991913,4cyclictest23980-21sh09:54:262
9018991913,4cyclictest16048-21sh12:13:162
9018991913,4cyclictest14085-21sh09:22:272
9018991913,3cyclictest29012-21sh09:10:012
9018991913,3cyclictest26181-21sh10:20:052
9018991912,5cyclictest20634-21grep07:35:152
9018991912,4cyclictest481-21sh12:01:032
9018991911,6cyclictest9816-21ls10:55:252
9018991911,6cyclictest9522-21basename08:25:012
9018991911,6cyclictest8278-21netstat10:30:202
901899191,16cyclictest28795-21diskmemload12:31:202
901899191,16cyclictest28795-21diskmemload12:31:202
9018991911,6cyclictest278-21systemd-journal12:39:552
901899191,16cyclictest22341-21cron08:55:002
9018991911,6cyclictest11808-21if_enp3s007:15:162
9018991910,7cyclictest29856-21date11:10:212
9018991910,6cyclictest13909-21head07:20:142
901799192,12cyclictest75-21kswapd010:10:111
901799192,12cyclictest20195-21apt-get07:35:121
9017991912,5cyclictest8424-21sed08:20:181
9017991912,5cyclictest20705-21expr08:50:131
9017991912,5cyclictest20584-21seq09:51:001
9017991912,4cyclictest8157-21sh09:18:171
9017991912,4cyclictest28795-21diskmemload11:40:061
9017991912,4cyclictest28528-21sh11:58:231
9017991911,6cyclictest21970-21seq09:28:371
9017991911,6cyclictest11923-21cat11:45:241
9017991910,6cyclictest31608-21tune2fs08:00:141
9017991910,6cyclictest278-21systemd-journal10:04:561
9017991910,6cyclictest1618-21cut08:05:181
9016991914,3cyclictest2788-21pmu-power10:50:210
9016991913,4cyclictest636-21runrttasks09:10:130
9016991913,4cyclictest18582-21sh11:02:120
9016991913,4cyclictest11946-21cpuspeed_turbos08:30:120
9016991912,5cyclictest26119-21seq10:44:100
9016991912,5cyclictest14263-21expr08:35:130
9016991911,6cyclictest31859-21tr12:00:220
9016991911,6cyclictest13347-21seq09:21:420
9016991911,5cyclictest5422-21acpi08:15:090
41932190,2sleep30-21swapper/309:15:163
265252190,2sleep00-21swapper/011:56:210
901999188,7cyclictest27446-21cut10:20:233
901999188,7cyclictest22507-21ssh11:05:163
901999182,12cyclictest0-21swapper/311:50:113
901999182,11cyclictest29870-21hddtemp_smartct09:10:173
901999181,4cyclictest0-21swapper/312:21:163
901999181,4cyclictest0-21swapper/309:42:543
9019991813,3cyclictest26854-21sh09:55:343
9019991812,4cyclictest34-21ksoftirqd/309:10:013
9019991811,5cyclictest5723-21sed12:30:003
9019991811,5cyclictest5293-21date08:15:013
901999181,15cyclictest34-21ksoftirqd/309:02:353
9019991811,5cyclictest31844-21sendmail_mailst12:00:223
9019991811,5cyclictest23227-21cat07:40:183
9019991811,5cyclictest19484-21grep07:30:243
9019991811,5cyclictest17083-21cat09:25:123
9019991811,5cyclictest16448-21expr08:40:143
9019991810,6cyclictest31838-21cat08:00:163
9019991810,6cyclictest28130-21sed10:45:173
9019991810,6cyclictest1086-21tune2fs10:25:163
901899189,6cyclictest20774-21cat10:40:122
901899189,6cyclictest16215-21grep07:25:152
901899187,3cyclictest141rcu_preempt11:00:132
901899182,11cyclictest24767-21apt-get09:00:142
901899182,11cyclictest18329-21apt-get08:45:122
901899182,11cyclictest18329-21apt-get08:45:122
9018991813,3cyclictest7866-21sh11:18:522
9018991813,3cyclictest29366-21sh11:35:052
9018991813,3cyclictest16168-21/usr/sbin/munin08:40:162
9018991813,3cyclictest14019-21acpi08:35:112
9018991812,4cyclictest636-21runrttasks12:23:132
9018991812,4cyclictest17682-21sh10:37:242
9018991812,4cyclictest10121-21sh10:07:522
9018991811,5cyclictest12018-21expr08:30:122
9018991810,6cyclictest1869-21cat10:50:122
901799189,7cyclictest560-21dbus-daemon08:15:001
901799189,7cyclictest28440-21users07:50:251
901799187,9cyclictest13796-21awk08:35:001
901799186,10cyclictest22-21ksoftirqd/111:01:281
901799184,10cyclictest27337-21/usr/sbin/munin10:45:211
9017991812,4cyclictest25854-21cron10:20:001
9017991812,4cyclictest14448-21sh10:59:491
9017991811,5cyclictest5651-21expr08:15:121
9017991811,5cyclictest27062-21expr09:05:131
9017991811,5cyclictest20087-21awk12:15:251
9017991811,5cyclictest14670-21cat08:35:171
9017991811,5cyclictest11846-21date08:30:001
9017991811,4cyclictest28795-21diskmemload09:13:471
901799181,14cyclictest20227-21munin-run08:50:001
901799181,14cyclictest20227-21munin-run08:50:001
9017991810,6cyclictest29924-21tune2fs11:35:141
9017991810,6cyclictest14679-21cat10:35:171
9017991810,6cyclictest10084-21sed11:20:151
901799180,16cyclictest19765-21ssh11:51:161
9016991813,3cyclictest30873-21sh12:24:320
9016991813,3cyclictest12526-21sh11:21:470
9016991812,4cyclictest22470-21sh11:54:000
9016991812,4cyclictest18905-21sh09:25:310
9016991812,4cyclictest16114-21fschecks_time07:25:150
9016991811,5cyclictest27657-21cat10:45:130
9016991811,5cyclictest22124-21seq08:52:130
9016991811,5cyclictest20929-21sed07:35:170
9016991810,6cyclictest22723-21expr07:40:130
9016991810,3cyclictest5860-21sh12:30:000
9016991810,3cyclictest5860-21sh12:30:000
901999172,11cyclictest15713-21apt-get07:25:123
901999172,10cyclictest31686-21apt-get10:00:113
9019991711,4cyclictest7322-21sshd08:20:003
9019991710,5cyclictest8884-21wc08:20:233
9019991710,5cyclictest5195-21sed11:40:233
9019991710,5cyclictest28910-21date07:55:003
9019991710,5cyclictest14284-21expr08:35:123
901899179,6cyclictest30375-21cat07:55:242
9018991712,3cyclictest32362-21mailstats08:00:222
9018991710,5cyclictest9958-21expr08:25:142
9018991710,5cyclictest1534-21cat08:05:162
901799179,6cyclictest16968-21sed08:40:191
901799179,6cyclictest1398-21grep10:25:191
901799179,5cyclictest0-21swapper/111:10:101
901799172,13cyclictest10925-21cut09:20:151
901799172,12cyclictest31270-21cron10:00:001
9017991712,3cyclictest26650-21idleruntime-cro09:05:001
9017991711,4cyclictest15242-21cron07:25:001
901699179,6cyclictest24704-21cat09:00:010
9016991710,5cyclictest9940-21expr08:25:140
9016991710,5cyclictest27040-21grep09:05:130
9016991710,5cyclictest18931-21basename08:45:160
9016991710,5cyclictest18931-21basename08:45:160
9016991710,5cyclictest14687-21cat07:20:230
901999162,11cyclictest0-21swapper/307:45:093
9019991611,3cyclictest9730-21/usr/sbin/munin08:25:233
901699160,14cyclictest30477-21grep07:55:240
9019991511,2cyclictest18304-21/usr/sbin/munin08:45:243
9019991511,2cyclictest18304-21/usr/sbin/munin08:45:243
901799146,6cyclictest16269-21which07:25:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional