You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-30 - 19:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat May 30, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1982724750,5sleep10-21swapper/107:05:201
2880921700,5sleep22148199cyclictest11:00:262
1257721020,2sleep00-21swapper/010:40:310
2147999951,75cyclictest2242-21aten_rbpower_vo09:50:210
2147999932,74cyclictest20279-21sed10:10:350
2147999921,6cyclictest11930-21grep08:10:350
2147999912,73cyclictest21399-21ssh12:10:170
2147999912,73cyclictest0-21swapper/011:50:180
2147999892,72cyclictest0-21swapper/012:35:240
203492890,2sleep00-21swapper/010:50:250
2147999871,62cyclictest0-21swapper/008:40:230
2147999862,60cyclictest0-21swapper/011:30:340
2147999853,69cyclictest23284-21/usr/sbin/munin10:15:350
2147999841,71cyclictest1165-21snmpd10:25:030
2147999832,69cyclictest0-21swapper/011:08:140
2147999832,66cyclictest0-21swapper/009:19:020
2147999832,64cyclictest0-21swapper/011:27:410
42462820,2sleep30-21swapper/309:51:593
2147999827,67cyclictest0-21swapper/008:50:120
2147999821,66cyclictest0-21swapper/010:35:220
26342810,5sleep12148099cyclictest10:29:331
26342810,5sleep12148099cyclictest10:29:331
2147999812,62cyclictest0-21swapper/012:01:480
2147999812,4cyclictest18251-21sh09:30:280
2147999805,62cyclictest9-21ksoftirqd/010:27:530
2147999805,62cyclictest9-21ksoftirqd/010:27:530
2147999801,59cyclictest0-21swapper/012:21:250
2147999801,59cyclictest0-21swapper/012:21:250
2147999801,3cyclictest0-21swapper/010:07:430
2147999791,62cyclictest0-21swapper/010:59:210
2147999782,64cyclictest0-21swapper/007:20:350
2147999782,61cyclictest0-21swapper/009:05:340
2147999772,60cyclictest0-21swapper/007:25:260
2147999771,65cyclictest19788-21ssh12:07:480
2147999767,52cyclictest9039-21apt-get08:05:190
2147999762,60cyclictest0-21swapper/011:22:000
2147999761,68cyclictest23759-21/usr/sbin/munin07:15:250
20612760,5sleep32148299cyclictest10:28:303
20612760,5sleep32148299cyclictest10:28:303
2147999752,61cyclictest0-21swapper/009:44:400
2147999752,59cyclictest0-21swapper/011:36:300
2147999751,64cyclictest0-21swapper/007:33:390
2147999751,64cyclictest0-21swapper/007:33:390
2147999751,4cyclictest106512sleep011:56:120
2147999751,4cyclictest0-21swapper/011:44:380
2147999751,4cyclictest0-21swapper/011:44:380
21479997426,38cyclictest7950irq/56-dwc_otg_10:00:350
2147999742,56cyclictest0-21swapper/009:00:060
2147999742,52cyclictest0-21swapper/011:05:010
2147999741,59cyclictest0-21swapper/011:10:420
2147999741,59cyclictest0-21swapper/011:10:420
2147999741,57cyclictest0-21swapper/009:13:500
2147999741,4cyclictest0-21swapper/008:37:420
2147999732,59cyclictest0-21swapper/009:22:180
2147999731,49cyclictest0-21swapper/012:15:330
2147999731,3cyclictest0-21swapper/009:39:030
2147999712,50cyclictest0-21swapper/009:02:290
2147999711,57cyclictest0-21swapper/009:49:090
2147999711,57cyclictest0-21swapper/009:49:090
2147999711,3cyclictest0-21swapper/007:54:340
2147999701,61cyclictest0-21swapper/012:25:330
2147999701,59cyclictest0-21swapper/011:45:240
2147999701,3cyclictest0-21swapper/011:19:350
2147999692,51cyclictest0-21swapper/008:32:090
2147999691,4cyclictest6945-21taskset07:59:510
2147999681,6cyclictest23215-21latency_hist07:15:060
2147999681,5cyclictest0-21swapper/009:59:000
2147999681,58cyclictest0-21swapper/010:47:550
2147999681,4cyclictest15766-21taskset09:27:360
21479996813,42cyclictest4741-21ssh10:30:500
261962670,1sleep30-21swapper/311:35:463
2147999675,44cyclictest0-21swapper/007:35:170
2147999671,3cyclictest0-21swapper/008:20:060
2147999668,49cyclictest0-21swapper/012:30:160
2147999661,50cyclictest7218-21/usr/sbin/munin08:00:170
309022650,2sleep30-21swapper/312:20:353
309022650,2sleep30-21swapper/312:20:353
2148299651,5cyclictest3257650diskmemload12:38:133
2147999641,4cyclictest0-21swapper/008:23:570
21482996314,38cyclictest17518-21/usr/sbin/munin09:30:293
2147999634,4cyclictest7950irq/56-dwc_otg_08:45:300
2147999634,4cyclictest7950irq/56-dwc_otg_08:45:300
45222620,2sleep20-21swapper/211:10:242
45222620,2sleep20-21swapper/211:10:242
2148299622,4cyclictest7950irq/56-dwc_otg_09:05:223
2148299622,4cyclictest28156-21ssh09:41:563
2148299621,5cyclictest8050irq/56-dwc_otg_08:35:063
2147999621,48cyclictest3112-21taskset07:46:590
2148299616,7cyclictest8050irq/56-dwc_otg_09:25:183
2148299611,42cyclictest7950irq/56-dwc_otg_08:40:343
21482996110,49cyclictest24481-21sed11:35:063
21480996110,9cyclictest7950irq/56-dwc_otg_08:10:181
2148299603,47cyclictest5665-21ssh09:15:323
2148299601,49cyclictest21017-21ssh12:10:033
2148299599,34cyclictest9231-21ssh10:36:293
2148299595,37cyclictest8050irq/56-dwc_otg_07:25:353
2148299591,5cyclictest25141-21apt-get07:20:173
2148299591,4cyclictest24469-21ls10:15:353
2148299591,47cyclictest11073-1kworker/u9:111:59:383
2148299591,46cyclictest14886-21rm10:05:173
2147999591,5cyclictest17753-21chrt08:29:270
2147999591,4cyclictest0-21swapper/007:43:570
2148299588,20cyclictest32349-21kworker/3:111:10:173
2148299588,20cyclictest32349-21kworker/3:111:10:173
2148299582,52cyclictest30667-21idleruntime-cro09:05:063
2148299573,4cyclictest7950irq/56-dwc_otg_08:05:233
2148299571,6cyclictest31316-21latency_hist10:25:073
2148299571,5cyclictest8050irq/56-dwc_otg_12:10:213
2148299571,4cyclictest0-21swapper/309:40:063
2148299571,39cyclictest7850irq/56-dwc_otg11:25:213
2148299566,32cyclictest7850irq/56-dwc_otg11:05:293
2148299565,34cyclictest8050irq/56-dwc_otg_07:35:313
2148299561,7cyclictest13468-21grep12:00:223
2148299561,4cyclictest8050irq/56-dwc_otg_09:22:253
2148299561,4cyclictest8043-21ssh09:56:103
35772550,1sleep10-21swapper/112:27:131
2148299555,39cyclictest32576-21diskmemload09:11:523
2148299553,3cyclictest7850irq/56-dwc_otg08:45:203
2148299553,3cyclictest7850irq/56-dwc_otg08:45:203
2148299552,38cyclictest0-21swapper/310:31:053
2148299551,5cyclictest29697-21ssh11:40:293
2148299551,5cyclictest29697-21ssh11:40:293
2148299551,4cyclictest8050irq/56-dwc_otg_12:30:193
2148299551,4cyclictest0-21swapper/311:55:063
2148299551,44cyclictest27316-21ssh10:59:063
2148299551,2cyclictest0-21swapper/311:15:173
2148299541,4cyclictest1308-21ssh11:45:263
2148299535,44cyclictest23341-21aten_rbpower_cu07:15:203
2148299531,5cyclictest0-21swapper/308:50:123
2148299531,4cyclictest20421-21ssh10:10:423
2148299531,4cyclictest0-21swapper/308:10:373
2148299523,45cyclictest0-21swapper/312:30:063
2148299522,5cyclictest5401-21/usr/sbin/munin07:55:253
2148299521,5cyclictest26154-21ssh12:15:263
2148299521,4cyclictest7850irq/56-dwc_otg07:50:383
2148299521,4cyclictest18432-21seq10:48:003
2148299521,47cyclictest2961-21unixbench_singl07:45:353
2148199523,9cyclictest7950irq/56-dwc_otg_08:00:172
2148299512,46cyclictest10748-21wget10:00:193
2148299511,8cyclictest13105-21ntpq11:20:283
21482995116,32cyclictest28143-21ssh11:00:113
2148299511,46cyclictest20016-21aten_rbpower_en10:50:213
2148299501,5cyclictest21824-21fschecks_time07:10:253
2148299501,5cyclictest0-21swapper/308:35:213
2148299501,4cyclictest30735-21ssh09:45:253
2148299501,4cyclictest30735-21ssh09:45:253
21480995024,6cyclictest6501-21sed07:55:341
2148299494,9cyclictest8050irq/56-dwc_otg_08:55:183
2148299492,3cyclictest7850irq/56-dwc_otg10:45:023
2148299491,4cyclictest15012-21memory08:20:263
2148299481,43cyclictest29876-21head07:30:343
2148299481,43cyclictest29876-21head07:30:343
2148299481,3cyclictest0-21swapper/307:40:173
21480994814,22cyclictest0-21swapper/107:35:121
2148299472,43cyclictest12649-21basename08:15:193
2148199477,26cyclictest0-21swapper/210:15:132
2148099477,26cyclictest0-21swapper/107:45:191
2148299468,36cyclictest7319-21aten_rbpower_vo08:00:213
2148099467,25cyclictest0-21swapper/112:15:191
2148099463,7cyclictest8050irq/56-dwc_otg_09:55:171
2148299452,6cyclictest16933-21grep08:25:283
2148199456,24cyclictest5270-21kworker/u8:408:30:132
2148199452,7cyclictest0-21swapper/210:10:182
2148199448,26cyclictest0-21swapper/207:20:202
2148199447,23cyclictest0-21swapper/209:05:112
2148099443,26cyclictest8050irq/56-dwc_otg_10:10:211
21480994422,5cyclictest26402-21sh11:36:091
21480994415,14cyclictest7850irq/56-dwc_otg11:25:151
232462420,2sleep20-21swapper/209:35:472
2148199427,23cyclictest0-21swapper/210:30:172
21481994218,6cyclictest29481-21latency_hist12:20:072
2103224221,13sleep00-21swapper/007:07:470
2148199417,23cyclictest0-21swapper/207:30:192
2148199417,23cyclictest0-21swapper/207:30:192
21481994110,19cyclictest301ktimersoftd/212:35:182
2148099417,25cyclictest0-21swapper/108:30:191
21480994119,4cyclictest27111-21ssh10:58:431
21480994118,5cyclictest5836-21/usr/sbin/munin11:50:351
2148199405,20cyclictest8924-21ssh11:55:122
21481994023,4cyclictest20272-21fschecks_time10:50:242
2148099401,4cyclictest8050irq/56-dwc_otg_09:35:201
2148199391,9cyclictest17248-21ssh09:30:062
21481993917,13cyclictest7850irq/56-dwc_otg12:05:232
21481993915,5cyclictest31916-21fschecks_time10:25:232
21481993915,5cyclictest31916-21fschecks_time10:25:232
2148199391,16cyclictest0-21swapper/211:30:232
2148099399,16cyclictest0-21swapper/112:00:531
2148099397,25cyclictest0-21swapper/107:40:181
2148099397,23cyclictest0-21swapper/107:15:201
2148099393,17cyclictest0-21swapper/109:50:191
2148099392,7cyclictest0-21swapper/109:25:351
2148099392,6cyclictest0-21swapper/108:25:171
65232382,3sleep20-21swapper/209:16:232
2148199387,17cyclictest7950irq/56-dwc_otg_08:15:172
21481993815,6cyclictest30719-21apt-config09:05:062
2148099384,4cyclictest8050irq/56-dwc_otg_11:05:191
2148099381,5cyclictest7950irq/56-dwc_otg_12:14:121
2148099381,4cyclictest7850irq/56-dwc_otg07:10:201
21480993813,4cyclictest12030-21sh11:58:441
2148099381,15cyclictest7850irq/56-dwc_otg10:20:071
2148199375,21cyclictest31-21ksoftirqd/212:25:182
2148199371,26cyclictest11875-21ssh10:40:212
2148099378,26cyclictest0-21swapper/108:45:191
2148099378,26cyclictest0-21swapper/108:45:191
2148099373,3cyclictest7850irq/56-dwc_otg09:45:181
2148099373,3cyclictest7850irq/56-dwc_otg09:45:181
2148099372,3cyclictest0-21swapper/110:00:251
2148099371,5cyclictest7950irq/56-dwc_otg_11:45:161
2148099371,4cyclictest30189-21grep09:00:331
2148099371,24cyclictest12899-21uniq08:15:231
2148099371,18cyclictest7950irq/56-dwc_otg_09:15:271
2148199361,5cyclictest0-21swapper/209:46:242
2148199361,5cyclictest0-21swapper/209:46:242
21481993615,7cyclictest0-21swapper/212:20:172
21481993615,7cyclictest0-21swapper/212:20:172
2148199361,3cyclictest8050irq/56-dwc_otg_11:35:182
2148199361,26cyclictest3905-21cut07:50:262
2148199361,12cyclictest7950irq/56-dwc_otg_10:35:362
21481993611,13cyclictest0-21swapper/210:55:162
2148099366,27cyclictest0-21swapper/109:05:111
2148099364,5cyclictest8050irq/56-dwc_otg_11:30:171
21480993618,6cyclictest9265-21sh09:20:231
2148099361,5cyclictest7850irq/56-dwc_otg11:00:301
2148199358,12cyclictest7950irq/56-dwc_otg_07:35:192
21481993513,6cyclictest0-21swapper/212:10:222
2148199351,12cyclictest0-21swapper/209:10:362
21481993511,12cyclictest0-21swapper/209:20:252
2148099351,8cyclictest26965-21ssh09:40:311
21480993512,4cyclictest1133-21runrttasks09:32:481
21480993511,12cyclictest141-21systemd-journal07:22:481
67402341,4sleep20-21swapper/211:51:482
2148199348,16cyclictest31-21ksoftirqd/210:00:172
2148199346,4cyclictest0-21swapper/208:40:192
2148199345,15cyclictest7950irq/56-dwc_otg_09:30:212
2148199342,26cyclictest0-21swapper/211:05:202
21481993418,4cyclictest7250-21sh12:31:212
2148099348,5cyclictest8050irq/56-dwc_otg_08:45:061
2148099347,23cyclictest0-21swapper/108:50:111
2148099346,5cyclictest7850irq/56-dwc_otg10:20:331
2148099343,3cyclictest7850irq/56-dwc_otg10:40:121
2148099342,9cyclictest5317-21sed11:10:341
2148099342,9cyclictest5317-21sed11:10:341
21480993427,5cyclictest8077-21rm12:32:521
21480993417,9cyclictest13583-21tail11:20:341
2148099341,11cyclictest0-21swapper/112:23:581
2148099341,11cyclictest0-21swapper/112:23:581
2148199337,6cyclictest7850irq/56-dwc_otg07:25:202
2148199337,4cyclictest6330-21timerandwakeup07:55:332
2148199337,14cyclictest0-21swapper/208:20:192
2148199335,13cyclictest8050irq/56-dwc_otg_11:15:282
21481993324,7cyclictest31-21ksoftirqd/211:45:182
2148199331,6cyclictest13887-21ntp_states12:00:282
2148199331,4cyclictest10820-21/usr/sbin/munin08:10:252
2148199331,22cyclictest31-21ksoftirqd/208:55:162
21481993311,4cyclictest8050irq/56-dwc_otg_10:05:232
21481993310,4cyclictest0-21swapper/210:48:292
2148099339,14cyclictest9696-21tr12:35:191
2148099338,4cyclictest0-21swapper/111:42:341
2148099338,4cyclictest0-21swapper/111:42:341
2148099337,23cyclictest0-21swapper/108:00:201
2148099337,16cyclictest0-21swapper/112:05:191
2148099333,9cyclictest8050irq/56-dwc_otg_08:20:181
21480993325,5cyclictest221ktimersoftd/108:05:201
21480993319,10cyclictest0-21swapper/110:50:181
2148099331,14cyclictest7850irq/56-dwc_otg10:35:251
2148199327,15cyclictest0-21swapper/209:40:192
2148199325,16cyclictest0-21swapper/209:55:242
21481993211,7cyclictest0-21swapper/209:50:302
2148099327,15cyclictest0-21swapper/107:30:191
2148099327,15cyclictest0-21swapper/107:30:191
2148099327,12cyclictest8050irq/56-dwc_otg_08:40:071
2148099327,11cyclictest8050irq/56-dwc_otg_10:47:371
2148099326,13cyclictest7950irq/56-dwc_otg_10:10:071
2148099324,6cyclictest0-21swapper/107:50:211
2148099323,3cyclictest7850irq/56-dwc_otg08:55:221
21480993223,7cyclictest27329-21/usr/sbin/munin07:25:241
21480993214,5cyclictest1832-21sshd09:10:381
161532320,5sleep216140-21ssh11:25:062
2148199315,6cyclictest30136-21grep11:40:342
2148199315,6cyclictest30136-21grep11:40:342
2148199315,15cyclictest8050irq/56-dwc_otg_07:10:292
2148199313,4cyclictest7850irq/56-dwc_otg07:45:272
2148199311,5cyclictest662-21ntpq07:40:272
2148099317,5cyclictest21-21rcuc/110:30:211
2148099313,5cyclictest7950irq/56-dwc_otg_11:15:181
2148199307,14cyclictest0-21swapper/208:05:212
2148199306,6cyclictest20227-21cut08:35:232
2148199302,7cyclictest0-21swapper/211:25:182
2148199301,14cyclictest23366-21cut07:15:202
2148199291,5cyclictest8050irq/56-dwc_otg_08:25:182
2148199291,16cyclictest0-21swapper/210:20:372
2148199281,8cyclictest26462-21chrt08:50:342
202272284,15sleep30-21swapper/307:05:273
2148199261,18cyclictest0-21swapper/208:45:142
2148199261,18cyclictest0-21swapper/208:45:142
201322263,11sleep20-21swapper/207:05:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional