You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-29 - 18:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Fri May 29, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29502998615,26cyclictest0-21swapper/009:50:180
2950299857,35cyclictest29573-21ssh11:40:180
241962810,2sleep20-21swapper/210:55:372
42462790,5sleep12950399cyclictest11:09:571
150392750,5sleep22950499cyclictest10:07:582
29502997424,10cyclictest0-21swapper/011:50:180
29502997424,10cyclictest0-21swapper/011:50:180
29502997215,32cyclictest9-21ksoftirqd/011:30:180
197282700,1sleep20-21swapper/208:35:352
110402680,5sleep32950599cyclictest09:25:233
2950499674,11cyclictest0-21swapper/209:55:172
2950299677,52cyclictest0-21swapper/012:15:180
2950299671,37cyclictest6468-21aten_rbpower_po09:20:190
29504996523,26cyclictest6510-21/usr/sbin/munin10:35:302
29503996410,4cyclictest0-21swapper/111:02:471
2950299642,30cyclictest25264-21ssh10:20:170
2950499638,13cyclictest7850irq/56-dwc_otg12:00:172
2950299637,43cyclictest0-21swapper/011:45:180
29502996217,6cyclictest0-21swapper/011:35:160
2950399618,4cyclictest0-21swapper/112:36:311
2950399617,5cyclictest27190-21/usr/sbin/munin09:00:261
2950299615,43cyclictest0-21swapper/010:45:050
2950399601,7cyclictest30407-21aten_rbpower_vo12:20:191
29503996010,6cyclictest31559-21sed09:10:361
29502996014,24cyclictest30266-21systemd-cgroups11:01:090
2950499595,29cyclictest0-21swapper/212:35:152
2950399597,17cyclictest0-21swapper/110:20:181
2950399596,5cyclictest7850irq/56-dwc_otg12:00:221
2950399591,4cyclictest7850irq/56-dwc_otg11:25:231
29502995937,5cyclictest8050irq/56-dwc_otg_12:25:270
2950299593,14cyclictest5653-21ssh10:35:170
29502995920,9cyclictest13-21rcuc/010:30:170
2950299591,31cyclictest0-21swapper/011:15:150
29504995818,24cyclictest6563-21cpuspeed09:20:192
29503995826,20cyclictest221ktimersoftd/111:55:181
29502995832,3cyclictest0-21swapper/012:20:330
2950299582,7cyclictest28096-21munin-run09:05:050
2950299582,35cyclictest28048-21sort09:45:260
29504995725,15cyclictest15374-21ssh11:22:332
29504995725,15cyclictest15374-21ssh11:22:332
29504995722,16cyclictest4682-21sh11:49:242
29504995721,14cyclictest2775-21chrt09:53:532
29504995720,12cyclictest11926-21sed09:25:352
2950399577,7cyclictest0-21swapper/112:19:361
2950399571,27cyclictest15241-21cut09:30:221
2950299574,24cyclictest141-21systemd-journal12:05:170
29502995730,17cyclictest24047-21unixbench-2d10:55:340
2950299572,7cyclictest1133-21runrttasks09:15:060
2950299572,5cyclictest0-21swapper/009:30:180
2950299572,30cyclictest0-21swapper/010:15:310
2950299571,14cyclictest10105-21latency_hist12:35:060
29504995623,11cyclictest21908-21rm10:54:052
29504995623,11cyclictest21908-21rm10:54:052
29504995622,21cyclictest9468-21ssh11:55:092
29504995621,20cyclictest29632-21ssh12:19:322
2950499562,11cyclictest0-21swapper/211:25:172
29504995618,21cyclictest24064-21runrttasks11:33:162
2950399563,5cyclictest7850irq/56-dwc_otg10:10:281
2950299566,46cyclictest23464-21ssh09:40:190
29502995628,7cyclictest11661-21grep12:35:330
2950299561,28cyclictest9750-21cut08:00:220
29504995521,16cyclictest7850irq/56-dwc_otg12:05:382
29504995517,22cyclictest8050irq/56-dwc_otg_12:30:292
29503995514,9cyclictest2663-21ssh09:53:421
2950299558,25cyclictest0-21swapper/007:50:060
2950299551,28cyclictest19511-21fschecks_time09:35:230
29502995510,27cyclictest13449-21timerandwakeup10:05:330
2950499543,9cyclictest0-21swapper/210:20:162
29504995427,4cyclictest16703-21aten_rbpower_en10:10:202
29504995425,17cyclictest17792-21latency_hist08:30:062
29504995425,16cyclictest2181-21diskstats07:30:212
29504995420,18cyclictest9101-21ssh11:54:542
29504995420,18cyclictest9101-21ssh11:54:542
29504995420,18cyclictest25378-21/usr/sbin/munin11:35:222
2950299549,26cyclictest15276-21sh08:20:050
29502995430,5cyclictest8050irq/56-dwc_otg_09:25:160
29502995429,9cyclictest8050irq/56-dwc_otg_11:05:320
2950299541,44cyclictest0-21swapper/012:12:500
29502995410,25cyclictest0-21swapper/009:55:280
2950499537,22cyclictest7950irq/56-dwc_otg_12:20:172
29504995327,14cyclictest23520-21rm12:10:582
29504995324,11cyclictest21377-21ssh09:37:052
29504995321,11cyclictest550-21ssh11:44:272
29504995320,14cyclictest18425-21ssh09:34:232
29504995320,12cyclictest695-21ssh10:29:332
29504995320,10cyclictest27720-21/usr/sbin/munin09:45:222
29504995319,21cyclictest7950irq/56-dwc_otg_09:40:202
29504995319,21cyclictest7950irq/56-dwc_otg_08:05:212
29504995319,19cyclictest3727-21ssh10:32:212
29504995318,13cyclictest11256-21sed10:40:342
29503995314,5cyclictest1133-21runrttasks11:54:581
29503995314,5cyclictest1133-21runrttasks11:54:581
2950299537,33cyclictest0-21swapper/008:55:190
29502995330,5cyclictest0-21swapper/010:00:320
29502995327,5cyclictest22156-21grep08:45:290
2950299531,26cyclictest0-21swapper/007:20:180
2950299531,18cyclictest4769-21aten_rbpower_vo11:10:180
2950299523,27cyclictest0-21swapper/007:10:150
2950299523,24cyclictest0-21swapper/008:10:060
2950299521,30cyclictest17867-21grep10:10:340
29504995122,12cyclictest22436-21ssh10:15:482
29504995119,11cyclictest5860-21ssh12:30:032
29504995118,11cyclictest29901-21ssh11:01:002
29504995117,12cyclictest7950irq/56-dwc_otg_07:30:052
2950299515,23cyclictest18065-21switchtime11:25:310
29502995126,6cyclictest8050irq/56-dwc_otg_08:50:330
29502995124,11cyclictest116550snmpd11:59:250
2950299511,26cyclictest13443-21cron08:15:060
2950299511,26cyclictest13443-21cron08:15:060
29502995112,25cyclictest0-21swapper/007:40:160
29502995112,25cyclictest0-21swapper/007:40:160
29504995026,10cyclictest2227-21ssh11:06:162
29504995023,11cyclictest32319-21ssh09:11:522
29504995021,16cyclictest2322-21ssh09:15:212
29504995018,13cyclictest12836-21ls11:20:072
29504995016,15cyclictest7950irq/56-dwc_otg_10:49:142
29504995011,25cyclictest0-21swapper/207:43:272
29504995011,25cyclictest0-21swapper/207:43:272
2950299504,23cyclictest0-21swapper/010:55:060
2950299504,23cyclictest0-21swapper/010:55:060
2950299501,28cyclictest300822chrt10:25:270
29504994923,17cyclictest10193-21ssh10:01:572
29504994923,11cyclictest1035-21chrt07:24:422
29504994921,18cyclictest8195-21seq07:51:292
2950299491,5cyclictest14447-21ntp_kernel_pll_12:00:270
29504994816,13cyclictest26840-21cut09:00:222
29504994813,21cyclictest8050irq/56-dwc_otg_11:13:442
29503994823,4cyclictest21256-21fschecks_count08:45:181
29503994815,9cyclictest23-21ksoftirqd/111:35:151
2950299486,29cyclictest13247-21cut11:20:220
2950299486,29cyclictest13247-21cut11:20:220
2950499472,11cyclictest0-21swapper/208:45:142
29504994719,13cyclictest3761-21chrt07:35:172
29504994717,16cyclictest7950irq/56-dwc_otg_07:48:502
29503994724,4cyclictest11040-21munin-plugin-st08:05:061
2950399471,8cyclictest7850irq/56-dwc_otg09:45:201
29502994725,10cyclictest0-21swapper/009:15:300
29502994722,12cyclictest0-21swapper/010:45:200
2950299471,28cyclictest30105-21grep09:10:060
2950599463,33cyclictest8050irq/56-dwc_otg_08:50:163
29505994618,4cyclictest9829-21fschecks_count11:55:223
29504994624,11cyclictest31012-21taskset07:14:122
29504994614,11cyclictest20619-21chrt08:43:272
2950399467,24cyclictest0-21swapper/107:30:181
29502994617,17cyclictest0-21swapper/008:35:330
2950599457,24cyclictest0-21swapper/309:50:183
29504994518,12cyclictest10447-21ls08:00:322
29504994516,11cyclictest13093-21taskset08:10:422
29504994516,11cyclictest13093-21taskset08:10:422
29504994515,13cyclictest24603-21/usr/sbin/munin08:55:302
2950299454,38cyclictest6503-21kworker/0:008:29:470
72232440,5sleep32950599cyclictest11:12:493
2950599441,4cyclictest19559-21ssh10:50:313
2950599441,4cyclictest19559-21ssh10:50:313
29504994417,13cyclictest24327-21taskset08:54:412
29504994414,11cyclictest14935-21runrttasks08:16:132
29504994412,11cyclictest15842-21taskset08:24:372
29502994417,13cyclictest0-21swapper/007:53:540
2950299441,30cyclictest0-21swapper/008:23:230
290882443,35sleep00-21swapper/007:08:220
29504994317,15cyclictest31454-21chrt07:16:572
29504994315,10cyclictest29813-21chrt09:09:082
2950399438,25cyclictest0-21swapper/107:50:151
2950299434,16cyclictest7950irq/56-dwc_otg_07:55:110
29502994323,2cyclictest0-21swapper/007:30:170
2950299432,28cyclictest0-21swapper/008:32:120
2950399425,10cyclictest7950irq/56-dwc_otg_09:20:181
228642420,2sleep10-21swapper/108:50:191
29504994112,18cyclictest7950irq/56-dwc_otg_07:57:162
2950299412,28cyclictest0-21swapper/007:20:060
2950599402,3cyclictest8050irq/56-dwc_otg_11:20:063
29505994018,13cyclictest8050irq/56-dwc_otg_10:40:203
29504994017,12cyclictest18114-21seq08:32:312
2950399401,5cyclictest7950irq/56-dwc_otg_11:40:171
2950299401,29cyclictest1165-21snmpd08:43:460
25652400,1sleep30-21swapper/311:06:533
220582400,3sleep122052-21ntp_states11:30:301
170602400,1sleep10-21swapper/108:25:311
2950599399,17cyclictest6372-21ssh09:20:163
2950599397,23cyclictest0-21swapper/310:05:193
29505993916,14cyclictest5895-21latency_hist12:30:063
2950399397,24cyclictest0-21swapper/110:00:181
2950399396,20cyclictest0-21swapper/110:40:161
29503993927,10cyclictest221ktimersoftd/108:35:151
29503993919,6cyclictest3908-21aten_rbpower_po09:55:201
2950599389,13cyclictest8050irq/56-dwc_otg_11:25:303
2950599385,19cyclictest7950irq/56-dwc_otg_11:35:163
29505993820,10cyclictest10332-21ssh10:02:123
2950399383,21cyclictest0-21swapper/109:35:281
2950399381,8cyclictest23454-21ssh09:40:181
2950299381,24cyclictest0-21swapper/007:28:400
2950599379,7cyclictest0-21swapper/311:45:223
2950599373,11cyclictest0-21swapper/308:05:153
29505993717,5cyclictest28184-21sh12:16:553
2950599371,4cyclictest8050irq/56-dwc_otg_10:25:193
2950399375,22cyclictest10674-21ssh09:25:191
29503993716,15cyclictest8050irq/56-dwc_otg_12:05:181
29502993718,9cyclictest0-21swapper/007:39:210
2950599367,14cyclictest8050irq/56-dwc_otg_11:00:253
2950399369,15cyclictest0-21swapper/110:50:181
2950399369,15cyclictest0-21swapper/110:50:181
29503993611,6cyclictest3314-21grep09:15:331
2950599358,13cyclictest381ktimersoftd/310:56:293
2950599356,20cyclictest0-21swapper/311:20:193
2950599356,20cyclictest0-21swapper/311:20:193
2950599353,21cyclictest0-21swapper/312:35:003
29505993523,5cyclictest329-21ssh09:12:423
2950599351,6cyclictest19501-21tune2fs09:35:233
29505993512,13cyclictest16840-21cat10:10:213
2950399351,3cyclictest8050irq/56-dwc_otg_11:10:171
2950399351,23cyclictest14941-21missed_timers10:45:271
2950399351,17cyclictest2574-21ssh11:45:341
29503993511,5cyclictest2945-21/usr/sbin/munin12:25:311
2950399351,14cyclictest21939-21/usr/sbin/munin12:10:321
2950599349,10cyclictest0-21swapper/311:40:193
2950599343,18cyclictest0-21swapper/312:11:133
29505993418,6cyclictest381ktimersoftd/309:55:183
29505993414,6cyclictest29918-21idleruntime-cro09:10:053
2950599341,25cyclictest30410-21ls07:10:293
2950399347,17cyclictest0-21swapper/110:25:211
2950399343,6cyclictest0-21swapper/111:17:401
29503993413,6cyclictest23490-21ssh10:55:271
2950399341,24cyclictest30511-21timerwakeupswit07:10:301
29503993410,12cyclictest8050irq/56-dwc_otg_11:21:371
29503993410,12cyclictest8050irq/56-dwc_otg_11:21:371
2950599339,6cyclictest23777-21cut09:40:233
2950599337,12cyclictest8050irq/56-dwc_otg_12:07:473
29505993318,6cyclictest15215-21fschecks_count09:30:223
2950599331,25cyclictest6407-21ls11:50:323
2950599331,25cyclictest6407-21ls11:50:323
2950599331,19cyclictest7850irq/56-dwc_otg10:15:203
2950399339,6cyclictest7850irq/56-dwc_otg10:10:061
2950399339,12cyclictest23-21ksoftirqd/112:33:391
29503993312,13cyclictest21-21rcuc/110:15:221
2950599327,15cyclictest0-21swapper/309:15:183
2950599321,14cyclictest8050irq/56-dwc_otg_08:20:053
2950399324,5cyclictest1462-21grep10:30:181
2950399322,20cyclictest0-21swapper/107:40:331
2950399322,20cyclictest0-21swapper/107:40:331
2950399321,11cyclictest8050irq/56-dwc_otg_10:35:181
2950599314,12cyclictest8050irq/56-dwc_otg_07:20:063
2950599312,5cyclictest6047-21fschecks_time10:35:233
29505993119,6cyclictest28328-21ssh10:23:363
2950599311,21cyclictest0-21swapper/310:48:213
2950399317,14cyclictest0-21swapper/107:55:121
2950599308,13cyclictest0-21swapper/312:02:553
2950599303,2cyclictest0-21swapper/311:30:193
2950599302,19cyclictest0-21swapper/312:20:183
2950599301,5cyclictest7950irq/56-dwc_otg_08:45:293
2950599301,5cyclictest10650-21df12:35:193
2950599301,22cyclictest0-21swapper/308:00:223
29505993011,4cyclictest0-21swapper/310:32:093
2950399305,6cyclictest7950irq/56-dwc_otg_08:15:321
2950399301,5cyclictest0-21swapper/108:35:061
2950599295,10cyclictest7950irq/56-dwc_otg_07:20:053
2950599293,23cyclictest8050irq/56-dwc_otg_08:25:173
2950599292,4cyclictest7850irq/56-dwc_otg07:30:053
2950599291,3cyclictest0-21swapper/308:55:163
2950599291,15cyclictest0-21swapper/308:43:023
2950399291,5cyclictest7850irq/56-dwc_otg09:06:261
2950599286,4cyclictest7950irq/56-dwc_otg_07:30:303
2950599284,18cyclictest7950irq/56-dwc_otg_09:04:523
2950599282,15cyclictest0-21swapper/307:35:333
2950599282,13cyclictest0-21swapper/309:45:153
2950599281,4cyclictest4892-21cat07:40:263
2950599281,4cyclictest4892-21cat07:40:263
2950599281,21cyclictest0-21swapper/308:34:463
2950599281,18cyclictest0-21swapper/307:49:223
2950599281,17cyclictest0-21swapper/308:35:193
2950599281,16cyclictest8050irq/56-dwc_otg_07:54:273
2950599281,12cyclictest7850irq/56-dwc_otg07:59:383
2950399286,12cyclictest0-21swapper/107:20:301
2950399283,13cyclictest8050irq/56-dwc_otg_07:29:581
2950399282,5cyclictest7850irq/56-dwc_otg08:05:281
2950399281,20cyclictest0-21swapper/107:17:031
2950399281,15cyclictest0-21swapper/108:45:061
288312284,15sleep20-21swapper/207:05:082
2950399273,12cyclictest0-21swapper/108:14:151
2950399273,12cyclictest0-21swapper/108:14:151
2950399271,5cyclictest4165-21munin-run07:40:061
288442273,13sleep10-21swapper/107:05:111
2950599262,6cyclictest0-21swapper/308:11:313
2950599262,6cyclictest0-21swapper/308:11:313
2950599261,10cyclictest0-21swapper/308:21:083
2950399262,5cyclictest5966-21apt-get07:45:121
2950399261,6cyclictest25083-21/usr/sbin/munin08:55:251
288842263,14sleep30-21swapper/307:05:423
2950399241,11cyclictest7850irq/56-dwc_otg08:23:491
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional