You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 00:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sun Aug 30, 2026 12:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
269862980,20sleep37950irq/56-dwc_otg_07:50:143
25082900,2sleep10-21swapper/108:10:321
299502890,2sleep10-21swapper/110:34:561
322792840,25sleep37950irq/56-dwc_otg_10:36:393
1270299832,68cyclictest0-21swapper/009:25:250
1270299821,54cyclictest0-21swapper/007:30:250
1270299812,54cyclictest21500-21kworker/u8:110:15:240
282962790,5sleep01270299cyclictest09:53:160
1270299763,61cyclictest0-21swapper/007:35:420
1270299762,49cyclictest12626-21grep09:35:130
315862740,3sleep131589-21latency11:15:341
12702997342,16cyclictest15628-21unixbench_multi07:15:410
12702997240,14cyclictest0-21swapper/008:10:410
12702997213,46cyclictest0-21swapper/009:45:130
12702997149,9cyclictest60-21kswapd011:30:190
12702997149,9cyclictest60-21kswapd011:30:190
12702997143,15cyclictest15737-21unixbench_multi10:55:420
12702997142,17cyclictest0-21swapper/007:10:280
1270299701,60cyclictest0-21swapper/008:40:430
1270299693,51cyclictest0-21swapper/009:50:130
1270299692,55cyclictest0-21swapper/012:05:440
1270299691,46cyclictest0-21swapper/010:30:250
243552680,1sleep10-21swapper/111:06:161
1270299684,46cyclictest16353-21taskset07:20:240
1270299681,56cyclictest0-21swapper/008:15:290
1270299681,54cyclictest20348-21fschecks_time12:20:300
1270499662,8cyclictest7750irq/56-dwc_otg08:40:272
1270299663,38cyclictest9-21ksoftirqd/011:25:120
1270299661,56cyclictest2944550ssh12:30:430
1270499654,42cyclictest7750irq/56-dwc_otg10:17:132
1270299654,51cyclictest0-21swapper/008:50:260
1270499645,41cyclictest7750irq/56-dwc_otg09:40:332
1270499642,43cyclictest7950irq/56-dwc_otg_07:10:422
1270499641,5cyclictest23492-21awk11:05:352
1270499641,4cyclictest23713-21ssh09:10:132
1270299645,46cyclictest0-21swapper/009:15:270
1270299641,55cyclictest16711-21idleruntime-cro09:40:120
1270299641,52cyclictest3340-21/usr/sbin/munin12:00:320
316042630,1sleep10-21swapper/108:00:381
12702996334,17cyclictest27135-21fschecks_count11:10:310
1270299632,48cyclictest0-21swapper/008:00:340
1270299631,43cyclictest0-21swapper/010:05:240
43662620,2sleep30-21swapper/310:03:383
1270499626,8cyclictest7750irq/56-dwc_otg09:00:252
1270499626,8cyclictest7750irq/56-dwc_otg09:00:252
1270499625,41cyclictest7950irq/56-dwc_otg_09:16:102
1270499625,17cyclictest2144-21ssh10:40:252
1270499623,5cyclictest7950irq/56-dwc_otg_11:45:272
1270499621,3cyclictest0-21swapper/212:20:392
12702996241,5cyclictest0-21swapper/012:15:260
1270299623,50cyclictest10706-21fschecks_time10:50:300
1270299623,50cyclictest10706-21fschecks_time10:50:300
1270299622,37cyclictest19746-21ls07:30:130
1270299621,49cyclictest27841-21fschecks_count11:50:310
1270299621,45cyclictest31307-21ps10:35:380
76202610,2sleep20-21swapper/212:05:272
44282610,2sleep20-21swapper/210:03:452
171952610,1sleep10-21swapper/110:19:261
1270499614,11cyclictest24134-21ssh12:25:252
12702996132,17cyclictest101ktimersoftd/009:05:120
12702996132,17cyclictest101ktimersoftd/009:05:120
1270299612,47cyclictest0-21swapper/011:35:250
1270299611,55cyclictest24892-21ntp_kernel_pll_12:25:360
1270299611,44cyclictest0-21swapper/011:15:360
12702996112,40cyclictest20037-21ntpq11:40:360
1270499607,38cyclictest7950irq/56-dwc_otg_10:45:362
1270499601,50cyclictest14158-21diskmemload09:35:472
1270299602,47cyclictest0-21swapper/009:05:360
1270299601,50cyclictest0-21swapper/010:10:290
1270299601,50cyclictest0-21swapper/009:20:330
1270499599,7cyclictest0-21swapper/211:20:232
1270499592,44cyclictest7850irq/56-dwc_otg_10:37:012
1270499591,6cyclictest21771-21apt-get10:25:192
1270299592,46cyclictest0-21swapper/011:27:280
1270499589,37cyclictest11250-21aten_rbpower_po08:35:272
1270499586,32cyclictest0-21swapper/211:30:152
1270499586,32cyclictest0-21swapper/211:30:152
1270499583,39cyclictest7850irq/56-dwc_otg_07:29:012
1270499583,39cyclictest0-21swapper/210:05:402
1270499582,6cyclictest18359-21rm11:00:132
1270499582,37cyclictest7750irq/56-dwc_otg11:41:502
1270499581,43cyclictest18521-21runrttasks11:00:152
12702995833,15cyclictest101ktimersoftd/010:45:300
1270299581,49cyclictest25225-21unixbench_singl09:10:430
203272570,3sleep2301ktimersoftd/210:22:372
1270499571,4cyclictest0-21swapper/209:59:402
1270299579,32cyclictest9-21ksoftirqd/010:00:390
1270299577,41cyclictest0-21swapper/012:10:390
1270299571,47cyclictest32664-21sort11:55:390
1270499566,32cyclictest0-21swapper/209:13:012
1270499563,42cyclictest7950irq/56-dwc_otg_11:39:412
1270499561,7cyclictest30986-21sshd11:15:272
1270499561,6cyclictest27634-21kernelversion07:50:302
1270499561,6cyclictest27634-21kernelversion07:50:302
1270499561,47cyclictest29096-21ssh11:51:082
1270499561,41cyclictest9598-21chrt08:30:302
1270499561,41cyclictest7950irq/56-dwc_otg_10:14:442
12704995613,32cyclictest27138-21uniq11:10:312
12702995620,4cyclictest0-21swapper/010:40:380
1270499558,35cyclictest0-21swapper/209:50:182
1270499551,6cyclictest6831-21cut11:25:262
1270499551,4cyclictest7750irq/56-dwc_otg07:50:142
1270499551,43cyclictest7107-1kworker/u9:112:12:182
1270499551,42cyclictest0-21swapper/211:59:412
12702995533,12cyclictest101ktimersoftd/011:45:400
1270299551,46cyclictest32628-21rm12:35:290
1270299551,46cyclictest0-21swapper/007:50:320
1270299551,46cyclictest0-21swapper/007:50:320
1270299551,46cyclictest0-21swapper/007:48:560
12702995511,5cyclictest9415-21cut08:30:270
1270499549,30cyclictest0-21swapper/212:35:202
1270499542,40cyclictest7850irq/56-dwc_otg_07:42:182
1270499542,3cyclictest7750irq/56-dwc_otg09:48:482
1270499541,5cyclictest7965-21ssh09:29:502
1270499541,5cyclictest29885-21ssh12:31:162
1270499541,4cyclictest5749-21apt-get08:20:192
1270499541,3cyclictest7950irq/56-dwc_otg_09:32:412
1270499541,2cyclictest0-21swapper/209:20:272
1270399547,27cyclictest0-21swapper/107:35:261
12702995427,15cyclictest0-21swapper/007:40:310
1270299542,42cyclictest0-21swapper/009:58:350
1270499535,38cyclictest7750irq/56-dwc_otg12:02:522
12704995317,33cyclictest18577-21seq08:55:272
1270499531,5cyclictest14308-21date10:55:142
1270499531,5cyclictest14308-21date10:55:142
1270499531,4cyclictest15788-21apt-get12:15:192
12704995310,40cyclictest17038-21ntpq07:20:342
12702995334,10cyclictest0-21swapper/008:35:250
1270299531,44cyclictest8386-21/usr/sbin/munin08:25:390
1270299531,44cyclictest0-21swapper/011:00:260
1270299531,33cyclictest13-21rcuc/011:05:230
1270599527,25cyclictest0-21swapper/307:35:263
1270499523,34cyclictest7950irq/56-dwc_otg_07:58:082
1270499521,3cyclictest0-21swapper/210:31:402
1270299521,41cyclictest23263-21timerandwakeup10:25:410
1270299521,41cyclictest0-21swapper/008:45:340
119092520,1sleep10-21swapper/107:05:411
1270599511,30cyclictest0-21swapper/309:00:303
1270599511,30cyclictest0-21swapper/309:00:303
1270499511,46cyclictest16596-21/usr/sbin/munin08:50:422
1270499511,3cyclictest0-21swapper/207:35:382
1270299511,42cyclictest5757-21/usr/sbin/munin08:20:320
1270299511,37cyclictest0-21swapper/010:20:180
1270499502,6cyclictest2361-21tune2fs08:10:302
12704995020,27cyclictest515-21fschecks_time08:05:302
1270499501,7cyclictest4993-21grep08:15:402
1270499501,5cyclictest7950irq/56-dwc_otg_08:30:132
1270399507,29cyclictest0-21swapper/108:30:271
1270299502,37cyclictest0-21swapper/008:58:320
12702995015,32cyclictest101ktimersoftd/007:55:480
1270299501,42cyclictest0-21swapper/008:10:080
1270499492,5cyclictest14924-21egrep08:45:262
1270499491,7cyclictest30839-21aten_rbpower_po08:00:272
1270499491,4cyclictest0-21swapper/207:15:262
1270499482,43cyclictest20361-21if_eth007:30:322
12703994825,5cyclictest752750wget12:05:261
1270399481,37cyclictest1311250wget08:40:281
1270399477,25cyclictest0-21swapper/111:50:271
1270399476,27cyclictest0-21swapper/109:00:261
1270399476,27cyclictest0-21swapper/109:00:261
121462473,36sleep00-21swapper/007:06:340
1270599467,23cyclictest0-21swapper/308:30:273
1270599457,24cyclictest0-21swapper/308:15:253
12703994521,15cyclictest17383-21ssh10:58:341
17132440,1sleep10-21swapper/110:00:301
1270399447,7cyclictest7850irq/56-dwc_otg_07:50:351
1270399447,7cyclictest7850irq/56-dwc_otg_07:50:351
12703994426,10cyclictest10751-21ssh09:32:161
1270399442,3cyclictest0-21swapper/109:25:241
1270599437,22cyclictest0-21swapper/311:50:273
1270599427,23cyclictest0-21swapper/308:20:193
1270599426,23cyclictest0-21swapper/307:10:273
12705994219,5cyclictest1180-21runrttasks12:31:133
1270599418,19cyclictest15343-21/usr/sbin/munin07:15:383
1270599415,24cyclictest4293-21ssh09:25:263
1270399413,8cyclictest15860-21ssh12:15:251
1270399412,6cyclictest7950irq/56-dwc_otg_07:45:261
12703994123,5cyclictest23-21ksoftirqd/111:39:461
12703994119,4cyclictest7140-21seq08:22:551
12703994118,5cyclictest23-21ksoftirqd/108:35:391
12703994116,5cyclictest23886-21sh09:47:581
12703994114,5cyclictest23589-21diskmemload11:21:141
172832400,4sleep311-21rcu_preempt08:50:353
1270599407,21cyclictest0-21swapper/311:30:273
1270599407,21cyclictest0-21swapper/311:30:273
1270599406,24cyclictest22129-21ssh10:25:193
1270599406,22cyclictest0-21swapper/308:55:263
152052390,2sleep10-21swapper/108:45:301
12703993916,7cyclictest5492-21sh10:05:171
1270399391,23cyclictest7950irq/56-dwc_otg_07:20:291
1270599388,7cyclictest7850irq/56-dwc_otg_12:25:363
1270599388,4cyclictest23589-21diskmemload10:47:043
12705993822,9cyclictest7663-21wget08:25:283
12705993822,9cyclictest25618-21ssh11:08:343
1270399387,9cyclictest7950irq/56-dwc_otg_12:05:131
1270399384,14cyclictest7950irq/56-dwc_otg_09:10:401
12703993822,9cyclictest2873-21ssh10:40:361
12703993818,5cyclictest5113-21seq08:16:311
12703993817,5cyclictest23-21ksoftirqd/111:45:321
12703993817,10cyclictest20488-21ssh11:01:431
12703993813,11cyclictest21498-21cron07:35:131
1270599373,19cyclictest0-21swapper/312:35:353
1270599372,21cyclictest0-21swapper/308:00:243
12705993719,4cyclictest1180-21runrttasks09:51:533
12705993712,5cyclictest28736-21latency_hist07:55:133
12705993712,5cyclictest28736-21latency_hist07:55:133
1270599371,22cyclictest0-21swapper/307:30:133
1270599371,20cyclictest1180-21runrttasks09:39:293
1270599371,15cyclictest12467-21latency_hist09:35:133
1270599366,21cyclictest0-21swapper/310:15:263
1270599363,10cyclictest0-21swapper/308:05:193
12705993615,13cyclictest39-21ksoftirqd/309:40:253
12705993615,12cyclictest6993-21kworker/3:012:08:223
1270399365,4cyclictest7750irq/56-dwc_otg08:55:141
1270399365,20cyclictest11066-21ssh11:30:271
1270399365,20cyclictest11066-21ssh11:30:271
1270399365,14cyclictest21-21rcuc/111:40:191
12703993627,5cyclictest28327-21aten_rbpower_vo09:15:271
1270399362,4cyclictest7950irq/56-dwc_otg_10:10:221
1270399361,22cyclictest31517-21systemd-journal07:18:511
1270599357,11cyclictest7750irq/56-dwc_otg10:50:363
1270599357,11cyclictest7750irq/56-dwc_otg10:50:363
1270599356,20cyclictest16242-21snmpd08:14:003
1270599352,7cyclictest0-21swapper/307:20:243
1270599351,7cyclictest2692350ssh11:10:273
12705993513,6cyclictest32539-21aten_rbpower_en09:20:293
1270399358,17cyclictest0-21swapper/110:37:111
1270399354,15cyclictest10647-21latency_hist11:30:131
12703993517,5cyclictest22032-1kworker/u9:211:10:291
1270399351,23cyclictest25738-21ls09:50:271
1270599349,5cyclictest3956-21latency12:00:333
1270599347,10cyclictest39-21ksoftirqd/308:35:273
1270599343,10cyclictest0-21swapper/309:59:553
12705993422,5cyclictest4958-21sh10:43:263
1270599342,18cyclictest0-21swapper/310:55:273
1270599342,13cyclictest0-21swapper/311:22:113
12705993418,8cyclictest381ktimersoftd/311:55:273
12705993416,10cyclictest10225-21rm11:29:333
12705993411,6cyclictest15637-21latency_hist12:15:143
1270599341,10cyclictest20248-21diskstats12:20:293
12705993410,14cyclictest0-21swapper/312:15:163
1270399345,3cyclictest0-21swapper/109:20:271
1270399342,3cyclictest7850irq/56-dwc_otg_10:50:261
1270399342,3cyclictest7850irq/56-dwc_otg_10:50:261
12703993414,7cyclictest0-21swapper/110:25:441
12703993412,4cyclictest23589-21diskmemload09:45:001
12703993412,10cyclictest0-21swapper/110:23:551
108962346,14sleep20-21swapper/207:05:272
1270599338,17cyclictest0-21swapper/310:20:393
1270599337,12cyclictest18962-21awk11:40:123
1270599336,12cyclictest7850irq/56-dwc_otg_10:14:393
1270599335,13cyclictest7750irq/56-dwc_otg09:45:333
12705993323,7cyclictest6863-21sh10:05:423
12705993317,9cyclictest381ktimersoftd/311:45:373
12705993313,10cyclictest0-21swapper/310:30:173
1270599331,21cyclictest39-21ksoftirqd/309:10:353
1270599331,18cyclictest16475-21ls08:50:143
1270599331,17cyclictest0-21swapper/307:40:503
1270399339,4cyclictest0-21swapper/111:57:231
1270399338,6cyclictest7850irq/56-dwc_otg_09:00:131
1270399337,6cyclictest0-21swapper/108:05:241
1270399337,11cyclictest0-21swapper/112:25:361
1270399333,17cyclictest7850irq/56-dwc_otg_10:45:311
1270399333,16cyclictest7950irq/56-dwc_otg_12:33:591
1270399331,9cyclictest8725-21grep08:25:421
12703993313,9cyclictest0-21swapper/112:11:141
1270599322,11cyclictest0-21swapper/311:15:333
12705993214,8cyclictest381ktimersoftd/311:42:483
1270599321,23cyclictest0-21swapper/309:18:443
1270599321,10cyclictest7950irq/56-dwc_otg_07:55:273
12703993218,4cyclictest221ktimersoftd/107:25:251
12703993215,5cyclictest21-21rcuc/109:35:331
1270399321,13cyclictest7950irq/56-dwc_otg_07:15:131
1270599315,15cyclictest39-21ksoftirqd/311:02:353
1270599311,6cyclictest14063-21grep08:40:423
1270399311,6cyclictest7950irq/56-dwc_otg_12:35:441
1270399311,16cyclictest23-21ksoftirqd/107:55:401
1270599301,19cyclictest0-21swapper/307:30:273
1270599301,14cyclictest7950irq/56-dwc_otg_09:05:423
1270399301,5cyclictest30289-21irqstats09:55:341
1270399301,3cyclictest7950irq/56-dwc_otg_09:05:391
1270399301,21cyclictest7950irq/56-dwc_otg_12:22:151
12703992923,4cyclictest23-21ksoftirqd/107:40:431
108752274,14sleep310877-21/usr/sbin/munin07:05:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional