You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 08:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Jun 23, 2026 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15511991217,61cyclictest0-21swapper/022:20:190
212012970,2sleep00-21swapper/020:55:070
323932880,6sleep11551299cyclictest21:15:411
323932880,6sleep11551299cyclictest21:15:411
156362800,3sleep339-21ksoftirqd/319:10:213
300532770,6sleep3381ktimersoftd/321:51:583
107632770,5sleep21551399cyclictest21:29:392
1551199766,59cyclictest0-21swapper/023:10:200
212662750,2sleep00-21swapper/023:00:070
15512997110,36cyclictest7950irq/56-dwc_otg_00:35:121
15512997014,19cyclictest13134-21ssh22:50:191
4172670,2sleep10-21swapper/123:52:261
15511996625,19cyclictest8050irq/56-dwc_otg_00:20:170
1551199656,28cyclictest26424-21ssh23:45:180
1551199652,55cyclictest0-21swapper/020:40:290
15511996333,13cyclictest8050irq/56-dwc_otg_20:20:080
150112633,12sleep20-21swapper/219:07:132
73962620,1sleep10-21swapper/122:04:321
73962620,1sleep10-21swapper/122:04:321
1551199621,45cyclictest0-21swapper/021:35:340
15512996116,25cyclictest6398-21ssh00:00:171
1551199609,40cyclictest0-21swapper/000:35:200
15511995915,33cyclictest13642-21latency_hist23:30:080
15511995913,26cyclictest27875-21/usr/sbin/munin00:25:290
1551199586,41cyclictest9-21ksoftirqd/021:15:340
1551199586,41cyclictest9-21ksoftirqd/021:15:340
15511995833,11cyclictest8050irq/56-dwc_otg_22:15:360
1551199572,42cyclictest0-21swapper/021:14:540
1551199567,38cyclictest0-21swapper/021:55:330
15511995645,5cyclictest7950irq/56-dwc_otg_22:30:350
1551199564,32cyclictest16686-21grep19:10:350
15511995624,3cyclictest0-21swapper/022:40:190
15511995616,5cyclictest26560-21sort21:10:070
15511995615,30cyclictest12996-21sh21:31:060
1551199561,45cyclictest10624-21mv22:07:400
1551199561,28cyclictest0-21swapper/022:00:280
1551199561,28cyclictest0-21swapper/022:00:280
1551499555,12cyclictest7950irq/56-dwc_otg_19:30:173
1551499552,14cyclictest0-21swapper/319:45:173
15512995526,20cyclictest0-21swapper/122:20:191
1551299552,16cyclictest0-21swapper/119:40:181
1551199558,34cyclictest11-21rcu_preempt00:05:070
1551199557,38cyclictest6778-21sh21:25:060
1551199552,27cyclictest10240-21latency_hist20:25:070
1551199551,45cyclictest0-21swapper/023:30:240
1551199551,34cyclictest24519-21latency_hist19:35:070
1551199542,31cyclictest26211-21latency_hist19:40:060
1551199541,50cyclictest0-21swapper/022:10:330
1551199541,45cyclictest0-21swapper/019:21:120
1551199541,30cyclictest31298-21ntp_kernel_err23:50:280
1551199541,28cyclictest5419-21latency_hist23:20:070
15511995410,36cyclictest0-21swapper/000:33:440
324562530,2sleep20-21swapper/219:55:282
1551299538,12cyclictest7850irq/56-dwc_otg00:10:201
1551299537,27cyclictest0-21swapper/121:00:201
1551299537,24cyclictest26314-21ssh22:25:501
1551199536,34cyclictest15011-21grep20:35:340
15511995335,5cyclictest1731-21timerandwakeup22:35:340
15511995320,20cyclictest22532-21munin-run19:30:070
15511995320,20cyclictest22532-21munin-run19:30:070
1551199531,29cyclictest22009-21latency_hist23:40:070
1551199527,28cyclictest20082-21unixbench_multi00:15:360
1551199522,29cyclictest6441-21latency_hist20:15:070
1551199522,28cyclictest4672-21latency_hist20:10:070
1551199521,45cyclictest0-21swapper/021:28:550
1551199521,40cyclictest28690-21ssh22:30:070
1551199521,22cyclictest25540-21/usr/sbin/munin23:05:260
15511995211,23cyclictest28711-21sh21:50:290
15512995131,5cyclictest10538-21fschecks_count20:25:241
1551199512,9cyclictest0-21swapper/019:45:060
15511995126,4cyclictest22608-21unixbench-2d23:00:360
1551199511,41cyclictest11725-21unixbench-2d00:05:350
1551499509,6cyclictest8050irq/56-dwc_otg_19:15:173
15512995013,25cyclictest0-21swapper/100:20:201
1551199502,7cyclictest0-21swapper/020:55:170
1551199501,32cyclictest3119-21ps23:55:310
1551499493,26cyclictest7850irq/56-dwc_otg00:05:193
1551299497,9cyclictest0-21swapper/120:20:121
15511994933,8cyclictest0-21swapper/021:40:320
15511994926,6cyclictest7950irq/56-dwc_otg_20:30:060
1551199491,5cyclictest24663-21proc_pri21:45:280
1551199491,28cyclictest18922-21sh19:20:070
1551399486,27cyclictest2876-21ssh21:20:202
1551299488,25cyclictest0-21swapper/120:00:201
1551299488,21cyclictest22546-21ssh00:19:541
1551299487,13cyclictest4773-21apt-get20:10:181
15512994814,25cyclictest8050irq/56-dwc_otg_00:05:191
1551199482,35cyclictest9096-21ssh22:45:210
1551199482,26cyclictest0-21swapper/022:50:280
1551199482,19cyclictest0-21swapper/020:00:230
1551499471,4cyclictest7850irq/56-dwc_otg23:55:073
1551299477,26cyclictest0-21swapper/123:10:121
1551299477,14cyclictest14103-21basename23:30:221
1551299474,27cyclictest25074-21/usr/sbin/munin21:05:251
1551299472,4cyclictest0-21swapper/119:35:191
15512994724,11cyclictest7850irq/56-dwc_otg19:22:381
15512994720,10cyclictest8050irq/56-dwc_otg_21:10:351
1551199479,25cyclictest0-21swapper/000:10:290
1551199472,7cyclictest24104-21grep21:00:350
1551199472,6cyclictest989-21latency_hist20:00:070
1551199472,29cyclictest0-21swapper/023:22:560
1551299469,20cyclictest9529-21latency_hist23:25:071
1551299469,20cyclictest8750-21cut22:45:071
1551299468,21cyclictest2032-21sh00:33:581
1551299467,25cyclictest0-21swapper/119:15:191
1551299463,24cyclictest18838-21rm20:45:501
15512994613,11cyclictest12639-21ssh23:28:181
15512994611,13cyclictest27279-21tr00:25:211
1551199461,23cyclictest0-21swapper/023:40:340
1551199461,23cyclictest0-21swapper/020:45:260
1551399457,25cyclictest0-21swapper/220:20:202
1551299458,21cyclictest9904-21/usr/sbin/munin22:45:331
1551299457,25cyclictest0-21swapper/122:05:131
1551299453,3cyclictest0-21swapper/123:40:201
1551299453,12cyclictest31832-21apt-config21:55:061
1551299453,12cyclictest31132-21unixbench_singl19:50:341
15512994513,13cyclictest27182-21ssh23:45:291
150902453,35sleep00-21swapper/019:08:080
1551499441,6cyclictest8050irq/56-dwc_otg_19:20:193
1551499441,22cyclictest24133-21cut21:45:213
1551299449,18cyclictest26402-21diskmemload21:24:191
1551299448,19cyclictest21832-21switchtime19:25:311
1551299448,19cyclictest21832-21switchtime19:25:311
1551299447,25cyclictest0-21swapper/120:35:191
1551299447,24cyclictest0-21swapper/123:55:191
1551299447,19cyclictest13814-21rm21:32:351
15512994426,8cyclictest8050irq/56-dwc_otg_22:32:531
1551299441,2cyclictest0-21swapper/123:00:211
15512994410,11cyclictest2826-21ssh22:37:031
15514994318,5cyclictest9307-21sh00:03:293
1551399436,23cyclictest0-21swapper/219:40:192
15513994320,6cyclictest10897-21ssh23:25:332
1551299438,27cyclictest0-21swapper/123:15:201
1551299437,13cyclictest7255-21awk20:15:281
15512994311,17cyclictest21511-21ssh23:39:211
15512994310,18cyclictest14095-21ssh22:11:271
1551199435,4cyclictest12463-21idleruntime20:30:250
15511994316,21cyclictest28885-21ntp_states19:45:290
1551299429,11cyclictest28219-21aten_rbpower_cu19:45:201
1551299427,23cyclictest125-21systemd-journal19:31:021
1551299427,20cyclictest25611-21ssh21:46:251
1551299427,17cyclictest12480-21taskset20:30:251
1551299423,5cyclictest8050irq/56-dwc_otg_21:55:191
15512994215,10cyclictest7950irq/56-dwc_otg_21:35:251
15512994211,18cyclictest7950irq/56-dwc_otg_21:29:531
30612410,2sleep20-21swapper/221:59:102
1551499413,28cyclictest39-21ksoftirqd/300:25:183
15514994122,5cyclictest23296-21sh00:20:233
1551299419,12cyclictest22681-21ssh21:43:391
1551299415,24cyclictest17017-21taskset20:40:411
15514994022,9cyclictest8965-21/usr/sbin/munin22:45:293
15514994015,7cyclictest7950irq/56-dwc_otg_23:00:213
15513994024,9cyclictest899-21ssh00:31:552
1551299407,21cyclictest16485-21ssh22:15:131
1551299407,16cyclictest20673-21ssh22:59:231
15511994016,12cyclictest727-21runrttasks19:51:180
190922390,5sleep31551499cyclictest22:17:473
1551399395,24cyclictest29827-21ssh23:10:202
1551399392,23cyclictest27035-21grep21:10:242
15513993917,5cyclictest24059-21timerandwakeup00:20:332
273372380,2sleep30-21swapper/321:10:283
1551499389,15cyclictest0-21swapper/321:25:193
15514993820,4cyclictest7097-21sh00:40:073
15513993819,4cyclictest1033650wget20:25:212
15513993816,5cyclictest31029-21if_eth023:50:252
15513993811,18cyclictest8050irq/56-dwc_otg_22:50:192
15512993814,5cyclictest0-21swapper/119:10:201
15512993813,7cyclictest0-21swapper/123:05:221
1551499379,6cyclictest0-21swapper/300:17:003
1551499374,6cyclictest8050irq/56-dwc_otg_20:05:203
15514993718,5cyclictest727-21runrttasks21:38:343
15514993715,11cyclictest26402-21diskmemload23:38:333
1551499371,3cyclictest7850irq/56-dwc_otg23:40:183
1551399377,21cyclictest5968-21ssh22:40:332
1551399376,7cyclictest8050irq/56-dwc_otg_21:35:362
1551399376,21cyclictest0-21swapper/200:35:212
1551399371,26cyclictest0-21swapper/223:35:072
1551499361,14cyclictest7850irq/56-dwc_otg22:05:283
1551499361,13cyclictest7850irq/56-dwc_otg20:30:353
1551399368,19cyclictest25677-21ssh23:05:212
1551399362,16cyclictest15429-21ssh00:10:292
15513993612,11cyclictest12712-21taskset21:30:362
1551499357,25cyclictest0-21swapper/319:25:183
1551499357,25cyclictest0-21swapper/319:25:183
1551499357,23cyclictest0-21swapper/320:45:203
1551499354,8cyclictest8050irq/56-dwc_otg_22:20:173
1551499354,20cyclictest8050irq/56-dwc_otg_22:55:183
15514993520,7cyclictest7850irq/56-dwc_otg22:00:393
15514993520,7cyclictest7850irq/56-dwc_otg22:00:393
15513993521,4cyclictest727-21runrttasks22:01:342
15513993521,4cyclictest727-21runrttasks22:01:342
1551399351,20cyclictest19774-21ls20:50:232
100962350,3sleep20-21swapper/222:06:432
1551499348,6cyclictest7950irq/56-dwc_otg_23:05:253
1551499347,23cyclictest0-21swapper/320:40:123
1551499347,22cyclictest0-21swapper/320:20:123
1551499347,21cyclictest0-21swapper/321:40:193
1551499347,15cyclictest0-21swapper/320:25:213
1551499342,6cyclictest7950irq/56-dwc_otg_23:20:173
1551499341,7cyclictest10269-21ssh23:25:253
15514993413,13cyclictest17590-21ssh00:13:273
15514993412,4cyclictest0-21swapper/322:37:163
1551399345,6cyclictest8050irq/56-dwc_otg_00:02:322
15513993426,5cyclictest2646-21ssh21:20:082
15513993426,5cyclictest2646-21ssh21:20:082
1551399342,23cyclictest0-21swapper/223:03:592
1551399342,22cyclictest0-21swapper/222:55:202
1551399341,7cyclictest13855-21ssh00:09:132
15513993414,6cyclictest24862-21fschecks_time19:35:242
1551399341,19cyclictest15653-21wc20:40:072
1551299348,13cyclictest20702-21taskset20:51:361
1551299346,13cyclictest8050irq/56-dwc_otg_21:00:061
1551499338,4cyclictest7850irq/56-dwc_otg21:20:073
1551499338,4cyclictest7850irq/56-dwc_otg21:20:073
1551499333,22cyclictest0-21swapper/323:30:103
1551499332,5cyclictest0-21swapper/320:15:073
15514993312,5cyclictest2655350wget23:45:213
15514993311,4cyclictest0-21swapper/322:50:343
1551399337,14cyclictest0-21swapper/220:10:212
1551399334,8cyclictest7950irq/56-dwc_otg_23:22:112
1551399332,6cyclictest0-21swapper/220:08:392
15513993323,8cyclictest8050irq/56-dwc_otg_22:10:262
1551399332,23cyclictest0-21swapper/219:12:522
1551399332,22cyclictest0-21swapper/222:45:472
15513993312,6cyclictest24854-21aten_rbpower_cu22:25:182
1551399331,20cyclictest29785-21ls19:50:062
1551399331,20cyclictest19246-21ls00:15:252
1551399331,15cyclictest7850irq/56-dwc_otg21:43:522
1551299336,5cyclictest8050irq/56-dwc_otg_19:55:341
150072333,22sleep30-21swapper/319:07:103
1551499329,5cyclictest8050irq/56-dwc_otg_20:35:173
1551499329,14cyclictest0-21swapper/323:13:573
1551499328,4cyclictest0-21swapper/322:40:193
1551499325,20cyclictest32130-21ssh21:55:113
1551499322,16cyclictest24948-21aten_rbpower_vo21:05:233
1551499321,25cyclictest28778-21ls22:30:073
1551499321,14cyclictest7950irq/56-dwc_otg_23:55:353
1551399327,14cyclictest7850irq/56-dwc_otg21:45:192
1551399326,14cyclictest0-21swapper/221:50:202
1551399324,3cyclictest0-21swapper/223:47:062
15513993226,4cyclictest24343-21munin-plugin-st19:35:062
15513993215,5cyclictest29-21rcuc/220:55:272
15513993215,5cyclictest23842-21sh22:23:532
15513993214,6cyclictest18211-21aten_rbpower_en23:35:212
15513993210,5cyclictest8050irq/56-dwc_otg_23:20:062
1551499316,12cyclictest8050irq/56-dwc_otg_23:16:033
1551499313,3cyclictest8050irq/56-dwc_otg_19:55:193
15514993114,5cyclictest21750-21cat20:55:273
1551499311,18cyclictest0-21swapper/319:53:403
15514993110,9cyclictest8050irq/56-dwc_otg_21:24:403
1551399317,6cyclictest0-21swapper/220:30:232
1551399317,5cyclictest7950irq/56-dwc_otg_22:15:352
1551399317,15cyclictest0-21swapper/222:30:202
1551399315,4cyclictest8050irq/56-dwc_otg_19:30:082
1551399315,4cyclictest8050irq/56-dwc_otg_19:30:082
1551399311,6cyclictest18906-21cron19:20:072
15513993113,10cyclictest7841-21resolvconf20:16:342
1551399311,16cyclictest7850irq/56-dwc_otg20:40:332
1551299310,13cyclictest0-21swapper/120:09:591
1551499307,14cyclictest0-21swapper/300:30:213
1551499305,16cyclictest0-21swapper/321:31:163
1551499302,15cyclictest0-21swapper/319:35:143
1551499301,10cyclictest16119-21ssh22:15:073
15514993010,13cyclictest23252-21diskstats21:00:233
1551399308,15cyclictest125-21systemd-journal19:51:062
15513993017,7cyclictest0-21swapper/200:28:472
15513993012,8cyclictest5668-21ssh23:59:232
1551399301,22cyclictest0-21swapper/222:38:232
15514992922,5cyclictest19484-21aten_rbpower_en20:50:203
1551499292,15cyclictest0-21swapper/322:30:123
15513992922,5cyclictest19439-21fschecks_count19:20:222
15513992910,4cyclictest0-21swapper/223:42:422
1551499286,6cyclictest7950irq/56-dwc_otg_20:16:553
1551499281,12cyclictest7950irq/56-dwc_otg_19:40:163
1551399282,19cyclictest0-21swapper/220:49:412
15513992813,4cyclictest301ktimersoftd/220:00:422
1551499274,13cyclictest8050irq/56-dwc_otg_20:00:393
15513992710,9cyclictest25799-21sed21:05:352
1551399265,11cyclictest0-21swapper/221:00:522
149442263,15sleep10-21swapper/119:06:231
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional