You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 08:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu Jul 16, 2026 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
85432225105,113sleep10-21swapper/119:07:261
8515217297,26sleep00-21swapper/019:07:030
2065821560,1sleep30-21swapper/323:54:063
84162145109,25sleep30-21swapper/319:05:463
8517214496,24sleep20-21swapper/219:07:052
2511421070,2sleep1211rcuc/121:30:531
874329256,27sleep00-21swapper/019:10:010
137192630,2sleep30-21swapper/320:35:173
270122560,1sleep00-21swapper/023:58:480
296942540,2sleep10-21swapper/121:10:231
280002530,2sleep00-21swapper/021:33:490
248662520,9sleep012-21ksoftirqd/019:45:180
71712500,1sleep10-21swapper/100:08:051
252912500,2sleep30-21swapper/322:20:153
901499480,35cyclictest25051-21rm00:20:440
6982480,1sleep10-21swapper/122:01:191
9016994714,18cyclictest14580-21ssh21:47:472
269402470,1sleep10-21swapper/123:34:331
9016994614,11cyclictest2048-21sh22:50:572
76452460,1sleep30-21swapper/321:18:393
17212460,1sleep00-21swapper/021:14:220
901499451,21cyclictest10977-21dump-pmu-power22:14:540
253752450,1sleep20-21swapper/223:32:572
130722450,1sleep00-21swapper/000:36:120
9016994414,18cyclictest28069-21diskmemload23:23:552
9016994410,18cyclictest18614-21ssh00:16:002
302572440,1sleep10-21swapper/123:36:101
901699439,24cyclictest0-21swapper/223:36:252
9016994311,18cyclictest6319-21sh21:17:192
901499431,31cyclictest6726-21ssh23:19:320
901499431,28cyclictest15197-21sh23:50:040
901499431,27cyclictest32389-21cron20:05:000
9014994312,22cyclictest28069-21diskmemload22:53:140
9016994214,16cyclictest7321-21ssh21:42:122
9016994214,11cyclictest5559-21ssh23:42:152
9016994214,11cyclictest20917-21/usr/sbin/munin23:05:222
9016994214,10cyclictest16395-21rm23:25:502
9016994212,9cyclictest7811-21sh00:08:442
9016994212,15cyclictest28069-21diskmemload22:19:312
9016994211,18cyclictest8051-21ssh22:07:022
901499424,24cyclictest2584-1kworker/u9:222:03:370
901499421,30cyclictest28069-21diskmemload22:59:410
9014994210,21cyclictest28069-21diskmemload21:23:580
200612420,2chrt0-21swapper/320:50:143
9016994115,10cyclictest30812-21ssh00:00:522
9016994114,12cyclictest11817-21ssh23:46:522
9016994114,10cyclictest28471-21ssh00:24:152
9016994114,10cyclictest26813-21ssh21:56:422
9016994114,10cyclictest15475-21ssh22:37:022
9016994113,11cyclictest9909-21ssh22:33:062
9016994113,11cyclictest5476-21taskset23:18:182
9016994113,11cyclictest16209-21ssh23:01:462
9016994112,10cyclictest18765-21ssh22:40:072
9016994111,19cyclictest27023-21ssh22:21:042
9016994111,17cyclictest13889-21ssh21:23:132
9016994111,12cyclictest26361-21ssh23:58:082
901499411,28cyclictest28069-21diskmemload23:21:010
901499411,28cyclictest28069-21diskmemload22:28:240
901499411,28cyclictest28069-21diskmemload21:27:240
901499410,29cyclictest6196-21sh21:41:030
9016994014,9cyclictest28069-21diskmemload22:57:052
9016994014,9cyclictest28069-21diskmemload22:10:522
9016994014,10cyclictest32034-21rm00:26:052
9016994013,9cyclictest10038-21ssh00:34:542
9016994013,9cyclictest10038-21ssh00:34:532
9016994013,16cyclictest15426-21ssh00:14:322
9016994013,10cyclictest25667-21chrt21:31:262
9016994012,15cyclictest17566-21chrt23:50:552
9016994012,11cyclictest32267-21rm21:36:242
9016994012,11cyclictest32267-21rm21:36:232
9016994012,10cyclictest18055-21chrt21:25:292
9016994011,12cyclictest31688-21sh23:13:592
9016994011,12cyclictest1532-21ssh21:14:112
9016994011,12cyclictest10718-21chrt19:12:182
9016994010,18cyclictest29642-21ssh22:47:502
901499405,16cyclictest8848-21cstates23:45:120
901499401,27cyclictest28069-21diskmemload22:35:220
901499401,27cyclictest28069-21diskmemload21:49:090
901499401,27cyclictest28069-21diskmemload21:17:260
901499401,21cyclictest28069-21diskmemload00:18:190
9014994010,16cyclictest636-21runrttasks19:40:100
901499400,29cyclictest28069-21diskmemload23:10:590
901499400,28cyclictest28069-21diskmemload22:33:530
9016993914,11cyclictest519-21rm22:01:112
9016993914,10cyclictest21305-21ssh21:52:532
9016993912,16cyclictest1697-21sh22:26:302
9016993910,9cyclictest0-21swapper/200:37:002
901499392,20cyclictest0-21swapper/020:50:120
901499391,27cyclictest28069-21diskmemload22:24:050
901499391,27cyclictest28069-21diskmemload22:19:140
901499391,26cyclictest4052-21ssh00:05:220
901499391,21cyclictest28069-21diskmemload22:43:280
901499391,21cyclictest28069-21diskmemload22:07:490
901499391,21cyclictest24289-21ssh23:31:520
901499391,18cyclictest28069-21diskmemload00:00:450
901499391,17cyclictest28069-21diskmemload23:07:150
901499390,28cyclictest28069-21diskmemload21:38:470
901499390,28cyclictest28069-21diskmemload21:38:460
901499390,27cyclictest28069-21diskmemload23:27:420
9016993812,9cyclictest6268-21chrt20:18:312
9016993811,9cyclictest23552-21chrt20:57:242
9016993811,16cyclictest32363-21taskset20:04:432
901499381,21cyclictest28069-21diskmemload00:30:320
901499381,21cyclictest28069-21diskmemload00:30:310
901499381,18cyclictest28069-21diskmemload23:36:250
901499380,27cyclictest28069-21diskmemload23:04:040
901499380,22cyclictest28069-21diskmemload21:55:050
901499380,18cyclictest15264-21latency_hist20:40:000
9016993712,8cyclictest10577-21taskset20:28:342
9016993712,10cyclictest17335-21chrt19:29:322
9016993711,9cyclictest27855-21chrt19:52:002
9016993711,9cyclictest13063-21taskset20:35:042
901499371,25cyclictest19694-21ssh21:51:140
901499371,18cyclictest28069-21diskmemload23:40:440
901499370,21cyclictest28069-21diskmemload00:25:580
901499370,21cyclictest21473-21taskset19:37:410
901499370,20cyclictest15215-21taskset19:24:460
9016993612,8cyclictest21365-21chrt19:36:182
901499361,21cyclictest28069-21diskmemload22:45:460
9016993510,9cyclictest19687-21cat19:35:022
901499351,21cyclictest11876-21ntp_states20:30:210
901499351,17cyclictest636-21runrttasks21:05:140
901499350,22cyclictest11705-21ssh00:10:440
9015993428,4cyclictest211rcuc/123:13:541
901499340,21cyclictest28115-21perf19:59:550
901499340,21cyclictest28115-21perf19:59:550
9016993312,8cyclictest17196-21taskset20:43:112
9015993329,3cyclictest211rcuc/121:15:151
9015993329,3cyclictest211rcuc/100:28:511
9015993329,3cyclictest211rcuc/100:18:231
9015993328,3cyclictest211rcuc/100:01:121
901499331,17cyclictest15438-21/usr/sbin/munin20:40:150
901499330,20cyclictest18437-21cat19:30:190
9015993228,3cyclictest211rcuc/121:37:111
9015993228,3cyclictest211rcuc/121:37:111
901699317,8cyclictest30072-21taskset19:58:012
901699317,8cyclictest30072-21taskset19:58:002
9015993127,3cyclictest211rcuc/122:16:131
9015993126,3cyclictest211rcuc/121:50:321
901499313,25cyclictest7634-21cut20:20:220
901499310,19cyclictest21637-21perf20:59:550
9015993027,2cyclictest211rcuc/123:41:301
9015993026,3cyclictest211rcuc/123:51:351
9015993026,3cyclictest211rcuc/123:08:471
9015993026,3cyclictest211rcuc/123:04:071
9015993026,3cyclictest211rcuc/122:31:451
9015993026,3cyclictest211rcuc/122:12:491
9015993026,3cyclictest211rcuc/122:09:481
9015993026,3cyclictest211rcuc/121:47:091
9015993026,3cyclictest211rcuc/120:53:111
9015993025,4cyclictest211rcuc/122:22:391
9015993010,7cyclictest22-21ksoftirqd/120:35:241
901699299,8cyclictest25687-21taskset21:02:052
9016992911,15cyclictest25956-21sh19:50:002
901599299,12cyclictest22-21ksoftirqd/120:15:001
9015992925,3cyclictest211rcuc/123:22:541
9015992925,3cyclictest211rcuc/123:19:411
9015992925,3cyclictest211rcuc/122:51:171
9015992925,3cyclictest211rcuc/122:47:091
9015992925,3cyclictest211rcuc/122:28:521
9015992925,3cyclictest211rcuc/121:22:211
9015992925,3cyclictest211rcuc/100:38:341
9015992925,3cyclictest211rcuc/100:14:211
901499291,17cyclictest636-21runrttasks20:49:220
901499291,17cyclictest636-21runrttasks20:49:210
901599286,7cyclictest22-21ksoftirqd/119:15:231
901599285,7cyclictest22-21ksoftirqd/120:30:191
9015992825,2cyclictest211rcuc/123:45:361
9015992824,3cyclictest211rcuc/122:56:511
9015992824,3cyclictest211rcuc/121:41:511
9015992824,3cyclictest211rcuc/100:30:531
9015992824,3cyclictest211rcuc/100:30:521
9015992824,3cyclictest211rcuc/100:20:291
9015992823,4cyclictest211rcuc/121:27:591
9015992823,3cyclictest211rcuc/122:40:331
9014992810,16cyclictest28095-21cron19:55:000
9016992711,13cyclictest12264-21grep20:30:232
901599277,11cyclictest22-21ksoftirqd/119:24:131
901599273,6cyclictest22-21ksoftirqd/121:55:251
9015992724,2cyclictest211rcuc/120:43:231
9015992723,3cyclictest211rcuc/123:55:471
9015992723,3cyclictest211rcuc/122:36:061
901499271,22cyclictest107552latency_hist20:29:590
319122270,2chrt0-21swapper/323:14:133
166522270,2sleep30-21swapper/323:26:043
107532270,17chrt34-21ksoftirqd/320:29:593
901799261,23cyclictest0-21swapper/319:10:483
9016992615,8cyclictest20170-21tune2fs20:50:162
9016992615,8cyclictest1170-21/usr/sbin/munin20:05:212
9016992615,6cyclictest66722sleep220:20:112
9016992614,9cyclictest22490-21hddtemp_smartct19:40:152
9016992614,9cyclictest13118-21awk19:20:002
9016992612,12cyclictest27209-21/usr/sbin/munin21:05:252
9015992622,3cyclictest211rcuc/120:45:101
9015992622,3cyclictest211rcuc/120:45:091
9015992621,4cyclictest211rcuc/120:05:041
901499261,22cyclictest5752-21awk20:15:240
280912260,1sleep30-21swapper/321:58:013
9016992516,6cyclictest2656-21cat20:10:112
9016992514,8cyclictest18099-21grep20:45:162
9016992514,8cyclictest18099-21grep20:45:162
9016992514,8cyclictest13461-21cut19:20:122
901599254,8cyclictest22-21ksoftirqd/120:00:211
9015992523,1cyclictest0-21swapper/119:10:541
9015992521,3cyclictest211rcuc/121:09:551
9015992520,3cyclictest211rcuc/120:56:091
9015992514,7cyclictest22259-21df19:40:121
9015992512,4cyclictest22-21ksoftirqd/123:26:391
901499252,18cyclictest32651-21apt-get20:05:120
901499251,21cyclictest2847-21tail20:10:150
901499251,21cyclictest24468-21fschecks_time21:00:130
901499251,21cyclictest17389-21munin-run19:30:000
22532250,4chrt2236-21ssh23:40:143
901799248,9cyclictest29660-21ssh00:25:123
9017992413,7cyclictest921-21cpu23:15:123
9015992420,3cyclictest211rcuc/119:45:111
901499241,20cyclictest0-21swapper/019:15:090
248832240,2sleep30-21swapper/322:44:473
901799232,15cyclictest9445-21ssh21:20:133
901799232,15cyclictest3877-21ssh21:40:133
9017992314,5cyclictest28069-21diskmemload23:55:093
9015992319,3cyclictest211rcuc/120:18:131
9015992318,4cyclictest211rcuc/119:57:381
9015992318,4cyclictest211rcuc/119:57:381
9015992318,4cyclictest211rcuc/119:27:141
9015992313,7cyclictest21283-21users19:35:271
316722230,2sleep30-21swapper/323:37:373
901599222,7cyclictest22-21ksoftirqd/120:28:331
901599221,7cyclictest22-21ksoftirqd/119:50:191
241992220,1sleep30-21swapper/300:20:223
9017992113,5cyclictest5848-21sed20:15:253
9017992113,4cyclictest4211-21sh00:05:233
9017992110,7cyclictest19140-21cut22:15:223
9015992115,4cyclictest211rcuc/119:31:451
901599210,7cyclictest24037-21apt-get21:00:111
901599210,5cyclictest8243-21taskset20:21:071
9017992014,4cyclictest22964-21sh21:54:343
9017992013,5cyclictest17601-21awk20:45:123
9017992013,5cyclictest17601-21awk20:45:123
9017992013,4cyclictest15237-21cron19:25:003
9017992012,6cyclictest16280-21sed20:40:193
9017992012,4cyclictest0-21swapper/323:45:213
9017992012,4cyclictest0-21swapper/322:05:113
9017992011,6cyclictest278-21systemd-journal20:25:013
9017991912,5cyclictest30902-21diskstats22:00:153
9017991912,5cyclictest13015-21sed22:35:183
9017991911,6cyclictest23038-21run-parts19:40:213
901799191,12cyclictest640-21snmpd00:00:223
901799188,6cyclictest23838-21grep21:30:203
901799187,9cyclictest0-21swapper/322:31:313
901799183,9cyclictest75-21kswapd022:50:113
901799183,9cyclictest15499-21apt-get19:25:103
901799182,11cyclictest17639-21apt-get19:30:133
901799181,8cyclictest21136-21ssh00:18:343
9017991813,3cyclictest9756-21sh00:34:373
9017991813,3cyclictest9756-21sh00:34:363
9017991813,3cyclictest9601-21sh22:56:503
9017991813,3cyclictest32092-21sh21:36:133
9017991813,3cyclictest32092-21sh21:36:123
9017991813,3cyclictest11621-21grep00:35:203
9017991812,4cyclictest19699-21sh23:05:053
9017991812,3cyclictest0-21swapper/322:10:233
9017991811,6cyclictest34-21ksoftirqd/322:30:013
9017991811,5cyclictest28160-21cut19:55:003
9017991811,5cyclictest13478-21expr23:00:123
9017991810,6cyclictest25346-21ls21:00:223
901799179,6cyclictest30311-21sed20:00:013
901799179,6cyclictest30311-21sed20:00:003
901799179,6cyclictest22510-21cat20:55:163
901799172,13cyclictest8247-21sed23:20:173
901799172,11cyclictest21006-21sh23:30:053
901799172,10cyclictest10896-21/usr/sbin/munin21:45:143
9017991712,3cyclictest28069-21diskmemload21:10:233
9017991712,3cyclictest27940-21sh22:46:063
9017991712,3cyclictest1145-21ntp_states20:05:203
9017991711,4cyclictest31404-21ntp_kernel_pll_20:00:213
901799171,11cyclictest16126-21ssh21:25:033
9017991710,5cyclictest15946-21wc00:15:003
9017991710,5cyclictest11228-21expr20:30:123
9017991710,4cyclictest19771-21cat19:35:113
901799169,5cyclictest26508-21df21:05:163
901799169,5cyclictest13199-21sed19:20:003
901799168,6cyclictest25707-21seq19:46:513
901799162,10cyclictest4270-21munin-run20:15:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional