You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 15:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed Aug 05, 2026 12:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32249991363,12cyclictest22527-21systemd-cgroups12:04:200
322499913112,92cyclictest125-21systemd-journal11:53:220
32249991302,94cyclictest0-21swapper/010:55:530
32249991302,94cyclictest0-21swapper/010:55:530
322499913010,87cyclictest0-21swapper/010:36:050
32249991273,101cyclictest0-21swapper/011:25:540
32249991272,90cyclictest0-21swapper/010:14:230
32249991265,91cyclictest0-21swapper/011:15:550
32249991244,102cyclictest0-21swapper/010:20:490
32249991244,102cyclictest0-21swapper/010:20:490
32249991193,91cyclictest14486-21kworker/u8:011:10:470
32249991192,99cyclictest0-21swapper/011:35:530
322499911914,78cyclictest7850irq/56-dwc_otg09:55:510
322499911912,97cyclictest12163-21uniq12:30:470
322499911912,97cyclictest12163-21uniq12:30:470
322499911725,66cyclictest18600-21ntpq10:40:530
32249991155,89cyclictest0-21swapper/010:50:590
32249991133,78cyclictest0-21swapper/011:40:480
32253991082,74cyclictest22542-21sd-pam12:04:213
322499910746,39cyclictest7950irq/56-dwc_otg_10:05:340
32253991026,74cyclictest8050irq/56-dwc_otg_10:14:273
32253991023,79cyclictest31015-21ssh10:55:543
32253991023,79cyclictest31015-21ssh10:55:543
32252991003,11cyclictest0-21swapper/210:20:502
32252991003,11cyclictest0-21swapper/210:20:502
3225399997,62cyclictest8050irq/56-dwc_otg_11:15:563
3225299992,10cyclictest31005-21ssh10:55:522
3225299992,10cyclictest31005-21ssh10:55:522
3224999984,78cyclictest9-21ksoftirqd/011:45:580
3225299966,9cyclictest8050irq/56-dwc_otg_09:55:532
3225399923,44cyclictest1747-21irqstats10:20:503
3225399923,44cyclictest1747-21irqstats10:20:503
3224999921,70cyclictest16480-21unixbench_singl11:55:570
32252999110,58cyclictest8050irq/56-dwc_otg_11:40:462
3225199893,57cyclictest1634-21ls10:20:481
3225199893,57cyclictest1634-21ls10:20:481
3225399883,62cyclictest13713-21ssh11:53:243
3225299884,14cyclictest12167-21fschecks_count12:30:482
3225299884,14cyclictest12167-21fschecks_count12:30:482
3225199882,55cyclictest8009-21ls11:45:591
3225299874,76cyclictest14881-21grep10:36:002
3225399863,76cyclictest3020-21ssh11:40:473
3225299855,8cyclictest8050irq/56-dwc_otg_10:51:002
3225299854,14cyclictest10619-21fschecks_time11:10:482
3225399843,70cyclictest19018-21apt-get10:05:353
3225399842,75cyclictest15030-21/usr/sbin/munin10:36:033
3225299837,10cyclictest14500-21diskmemload10:13:492
3225299833,11cyclictest14486-21kworker/u8:011:15:582
32252998312,65cyclictest20719-21vmstat12:01:072
3224999832,71cyclictest0-21swapper/012:05:540
3224999831,72cyclictest0-21swapper/011:20:420
3224999831,71cyclictest30433-21ssh09:36:060
3224999821,63cyclictest0-21swapper/012:15:520
32249998212,5cyclictest26817-21idleruntime-cro09:30:290
3225399813,71cyclictest18539-21netstat10:40:523
3225399812,72cyclictest7912-21ssh11:45:573
3225299818,66cyclictest8050irq/56-dwc_otg_11:53:222
3225299813,9cyclictest8050irq/56-dwc_otg_11:25:572
3224999811,66cyclictest16012-21/usr/sbin/munin10:00:550
3224999811,64cyclictest2839-21ssh11:00:540
3225399803,70cyclictest11063-21ssh11:10:553
3225399802,56cyclictest8050irq/56-dwc_otg_09:55:533
3224999802,69cyclictest16278-21ssh12:35:470
3225399793,67cyclictest8050irq/56-dwc_otg_11:25:563
3225399783,68cyclictest12406-21ssh12:30:523
3225399783,68cyclictest12406-21ssh12:30:523
3225399773,56cyclictest31565-21ssh11:35:503
32252997617,53cyclictest31577-21kernelversion11:35:502
3224999761,62cyclictest0-21swapper/012:25:490
129122760,5sleep13225199cyclictest11:52:051
3225299753,11cyclictest19365-21ssh10:05:342
3225299753,11cyclictest18587-21taskset10:40:532
3224999753,11cyclictest0-21swapper/007:10:320
3224999751,62cyclictest6947-21ssh09:50:370
3224999751,62cyclictest0-21swapper/012:22:540
3224999751,61cyclictest14500-21diskmemload10:30:290
3225399742,64cyclictest27223-21ssh10:50:593
3225299743,24cyclictest0-21swapper/211:45:582
32251997410,14cyclictest21-21rcuc/112:04:211
3224999741,61cyclictest18431-21ssh09:16:360
3224999735,14cyclictest0-21swapper/009:41:120
3224999731,57cyclictest0-21swapper/009:48:220
3225299722,9cyclictest32248-21cyclictest07:10:322
32251997233,10cyclictest23-21ksoftirqd/110:14:221
3224999721,52cyclictest0-21swapper/010:32:590
182092720,1sleep10-21swapper/111:59:011
39392710,1sleep20-21swapper/209:45:472
3225399711,52cyclictest15995-21ntpq10:00:543
3225199714,44cyclictest11099-21ps11:10:561
3224999711,59cyclictest750-21snmpd07:49:380
3224999711,59cyclictest750-21snmpd07:49:380
3224999711,15cyclictest0-21swapper/011:34:080
3224999701,57cyclictest16604-21rm09:13:490
3224999701,18cyclictest0-21swapper/009:33:400
32253996921,29cyclictest6449-21sh09:50:123
32251996924,28cyclictest7850irq/56-dwc_otg10:05:361
3224999691,52cyclictest0-21swapper/010:47:200
3225399681,57cyclictest4937-21ssh11:04:023
3224999681,55cyclictest30127-1kworker/u9:310:16:270
3224999681,55cyclictest0-21swapper/012:12:020
3224999681,54cyclictest0-21swapper/008:01:130
32251996729,19cyclictest221ktimersoftd/110:55:531
32251996729,19cyclictest221ktimersoftd/110:55:531
32251996724,17cyclictest7850irq/56-dwc_otg11:25:541
32251996713,36cyclictest518-21rs:main1
3224999672,49cyclictest0-21swapper/009:21:120
32249996715,41cyclictest7950irq/56-dwc_otg_11:06:440
32249996714,39cyclictest141212chrt09:05:510
32249996714,39cyclictest141212chrt09:05:510
32249996650,5cyclictest7850irq/56-dwc_otg07:30:290
3225399656,46cyclictest18627-21ssh11:20:443
3225199657,21cyclictest27296-21ssh10:51:001
3224999651,53cyclictest0-21swapper/008:12:020
3225199645,39cyclictest0-21swapper/111:40:471
3224999642,50cyclictest0-21swapper/008:51:450
32253996323,32cyclictest8050irq/56-dwc_otg_09:19:233
3225199637,19cyclictest0-21swapper/110:45:421
32251996325,18cyclictest8050irq/56-dwc_otg_09:55:511
32251996310,27cyclictest7850irq/56-dwc_otg12:30:491
32251996310,27cyclictest7850irq/56-dwc_otg12:30:491
3224999639,43cyclictest0-21swapper/007:39:010
3224999632,52cyclictest0-21swapper/007:31:210
3224999631,52cyclictest0-21swapper/008:23:070
3224999631,51cyclictest0-21swapper/008:46:090
3224999631,50cyclictest0-21swapper/007:55:160
3224999631,19cyclictest0-21swapper/007:20:200
207312630,3sleep120729-21grep09:20:291
3225399621,52cyclictest22053-21sh09:22:353
3225399621,51cyclictest8416-21rm10:29:053
3225399621,49cyclictest29972-21ssh12:12:443
3225399621,47cyclictest3173-21taskset07:38:593
3225299625,4cyclictest8050irq/56-dwc_otg_10:47:472
3225199623,19cyclictest15247-21ps11:16:001
3224999621,51cyclictest0-21swapper/008:17:350
3224999621,51cyclictest0-21swapper/008:17:350
3225399615,45cyclictest10884-21latency_hist09:55:293
3225399614,45cyclictest10100-21taskset08:34:303
3225399614,43cyclictest0-21swapper/312:26:423
3225399611,52cyclictest25714-21ssh12:07:333
3225399611,49cyclictest10569-21chrt08:37:403
3225399611,47cyclictest326902sleep312:15:553
3225299614,5cyclictest7950irq/56-dwc_otg_10:00:582
3225299611,6cyclictest16405-21unixbench-2d11:55:572
3224999613,45cyclictest13878-21latency_hist09:05:290
3224999611,52cyclictest0-21swapper/007:55:370
3224999611,44cyclictest0-21swapper/008:57:020
3224999611,18cyclictest0-21swapper/007:20:400
307312610,1sleep20-21swapper/212:14:042
3225399605,47cyclictest7950irq/56-dwc_otg_10:34:123
3225399601,49cyclictest8315-21seq08:20:223
3225399601,49cyclictest8315-21seq08:20:223
3225399601,47cyclictest0-21swapper/310:50:303
3225299606,3cyclictest8050irq/56-dwc_otg_12:36:372
3225299601,6cyclictest13939-21latency_hist09:05:302
3225299601,3cyclictest7950irq/56-dwc_otg_09:39:172
3225399595,49cyclictest0-21swapper/307:10:333
32252995918,38cyclictest27363-21if_eth011:30:492
3225399581,46cyclictest0-21swapper/309:28:093
3225299581,6cyclictest2992-21chrt07:36:412
3224999582,4cyclictest10976-21latency_hist08:40:300
3224999582,16cyclictest0-21swapper/008:40:310
3224999581,18cyclictest0-21swapper/008:06:280
3224999581,17cyclictest0-21swapper/007:44:390
57112570,2sleep10-21swapper/111:05:261
3225399575,42cyclictest7950irq/56-dwc_otg_12:24:143
3225399574,42cyclictest2942-21ssh09:44:243
3225399571,46cyclictest16803-21ssh09:14:103
3225399571,45cyclictest8023-21rm11:07:053
3225199578,34cyclictest0-21swapper/111:20:431
3225399566,40cyclictest11213-21taskset08:43:013
3225399565,41cyclictest11873-21runrttasks08:48:373
3225399562,44cyclictest0-21swapper/307:58:053
3225399562,43cyclictest750-21snmpd08:15:093
3225399561,43cyclictest285692chrt11:31:373
3225399561,35cyclictest30425-21sh09:36:063
32252995612,4cyclictest727-21runrttasks09:22:132
3225199569,28cyclictest221ktimersoftd/111:35:501
32251995620,13cyclictest18537-21ssh10:40:521
3224999561,45cyclictest0-21swapper/008:32:000
3225399551,44cyclictest16722-21ssh11:56:183
3225399551,43cyclictest0-21swapper/310:15:553
3225299556,6cyclictest8050irq/56-dwc_otg_10:18:182
3225299556,3cyclictest0-21swapper/212:21:582
3225299551,4cyclictest8050irq/56-dwc_otg_09:44:492
3224999551,13cyclictest0-21swapper/008:29:090
3225399544,40cyclictest0-21swapper/309:30:353
3225399544,36cyclictest61092chrt08:00:533
3225399541,41cyclictest1875-21chrt07:28:213
3225299545,5cyclictest8050irq/56-dwc_otg_11:21:442
3225299544,7cyclictest0-21swapper/210:25:402
3225299544,4cyclictest7950irq/56-dwc_otg_09:55:292
316392543,41sleep10-21swapper/107:06:441
316392543,41sleep10-21swapper/107:06:441
3225399531,5cyclictest0-21swapper/307:44:393
3225299531,4cyclictest12440-21ssh10:33:512
327652520,2sleep10-21swapper/110:19:281
3225399524,45cyclictest8948-21latency_hist08:25:293
3225399521,42cyclictest125632sleep308:54:353
3225399521,41cyclictest6793-21taskset08:06:483
3225299521,4cyclictest32654-21ntpq12:15:542
3225399513,35cyclictest10582chrt07:20:403
3225399511,39cyclictest125-21systemd-journal12:38:153
3225299512,4cyclictest7950irq/56-dwc_otg_11:04:042
3225299511,5cyclictest25513-21rm12:07:112
3225399501,4cyclictest0-21swapper/307:49:363
3225399501,4cyclictest0-21swapper/307:49:363
3225399501,36cyclictest13862-21runrttasks09:05:283
3225299501,3cyclictest8050irq/56-dwc_otg_07:20:432
3225399491,41cyclictest796-21chrt07:20:203
3225299495,8cyclictest17032chrt07:26:022
3225299492,6cyclictest14792-21ssh09:10:372
3225299492,3cyclictest8050irq/56-dwc_otg_09:20:262
32252994916,4cyclictest26941-21latency_hist09:30:302
3225299491,4cyclictest0-21swapper/209:34:022
3225399482,3cyclictest8050irq/56-dwc_otg_08:59:503
3225299482,3cyclictest7950irq/56-dwc_otg_12:26:042
3225399472,32cyclictest0-21swapper/307:52:473
3225399471,35cyclictest0-21swapper/307:33:593
3225299471,4cyclictest0-21swapper/211:06:512
3225399462,5cyclictest7950irq/56-dwc_otg_09:08:343
3225399462,5cyclictest7950irq/56-dwc_otg_09:08:343
3225299451,3cyclictest8050irq/56-dwc_otg_09:08:192
3225299451,3cyclictest8050irq/56-dwc_otg_09:08:192
3225399441,8cyclictest9705-21cat08:30:293
3225299442,5cyclictest301ktimersoftd/208:49:012
316332443,34sleep00-21swapper/007:06:390
316332443,34sleep00-21swapper/007:06:390
3225299431,16cyclictest7122-21grep08:10:292
3225199436,4cyclictest0-21swapper/110:25:421
3225199432,6cyclictest19830-21apt12:40:291
32251994115,16cyclictest0-21swapper/112:10:431
32252994015,4cyclictest301ktimersoftd/208:23:252
3225199409,4cyclictest0-21swapper/110:30:501
3225199408,25cyclictest26827-21ssh11:30:351
32251994016,6cyclictest11391-21sh12:30:141
320262406,20sleep332014-21ls07:10:313
320262406,20sleep332014-21ls07:10:313
3225199399,20cyclictest0-21swapper/110:00:441
3225199397,8cyclictest7850irq/56-dwc_otg07:10:321
32252993814,5cyclictest301ktimersoftd/207:45:292
3225199387,19cyclictest0-21swapper/111:08:171
32251993832,4cyclictest23543-21sh12:05:411
3225199369,5cyclictest125-21systemd-journal07:30:291
32251993626,7cyclictest26871-21latency_hist09:30:301
45012350,1sleep10-21swapper/109:46:461
3225199357,13cyclictest7950irq/56-dwc_otg_12:15:501
3225199355,18cyclictest7850irq/56-dwc_otg09:42:191
3225199352,21cyclictest0-21swapper/109:08:321
3225199352,21cyclictest0-21swapper/109:08:321
3225199345,9cyclictest27665-21ssh09:31:371
3225199332,5cyclictest8050irq/56-dwc_otg_12:20:421
3225199332,17cyclictest0-21swapper/109:50:471
32252993213,5cyclictest9691-21cut08:30:292
3225199323,5cyclictest4573-21taskset07:50:301
3225199323,5cyclictest4573-21taskset07:50:301
32251993217,4cyclictest13866-21cron09:05:291
32251993213,12cyclictest221ktimersoftd/109:40:101
32252993113,8cyclictest25252chrt07:33:392
3225199315,12cyclictest8050irq/56-dwc_otg_08:40:291
3225199313,8cyclictest125-21systemd-journal08:20:291
3225199313,8cyclictest125-21systemd-journal08:20:291
32251993110,4cyclictest18485-1kworker/u9:009:24:201
3225299301,15cyclictest750-21snmpd08:38:292
3225199307,14cyclictest0-21swapper/108:00:341
3225199305,6cyclictest0-21swapper/107:33:111
3225199302,13cyclictest0-21swapper/109:14:081
3225199299,11cyclictest0-21swapper/107:43:011
3225199291,6cyclictest7850irq/56-dwc_otg08:50:291
3225299281,4cyclictest9550irq/48-DMA15:26:472
32252992810,4cyclictest8050irq/56-dwc_otg_08:35:292
3225199281,6cyclictest5768-21munin-run08:00:291
3225199281,15cyclictest8050irq/56-dwc_otg_08:25:291
3225299272,4cyclictest7950irq/56-dwc_otg_09:00:292
32252992721,4cyclictest5992-21latency_hist08:00:302
3225299271,5cyclictest750-21snmpd07:18:132
3225199279,9cyclictest125-21systemd-journal07:38:491
3225199276,11cyclictest8050irq/56-dwc_otg_07:53:191
3225199274,4cyclictest0-21swapper/108:44:441
3225199272,4cyclictest0-21swapper/108:59:191
3225299261,3cyclictest7850irq/56-dwc_otg07:50:152
3225299261,3cyclictest7850irq/56-dwc_otg07:50:152
3225199263,5cyclictest0-21swapper/107:20:231
32251992619,5cyclictest12321-21seq08:51:301
3225199261,6cyclictest7950irq/56-dwc_otg_08:13:481
315592263,14sleep20-21swapper/207:05:402
315592263,14sleep20-21swapper/207:05:402
3225299252,3cyclictest7950irq/56-dwc_otg_08:51:142
3225299251,4cyclictest0-21swapper/208:19:412
3225299251,4cyclictest0-21swapper/208:19:412
3225199252,14cyclictest0-21swapper/107:24:421
3225299247,7cyclictest32248-21cyclictest07:51:542
3225299241,5cyclictest0-21swapper/208:12:582
3225299241,4cyclictest7950irq/56-dwc_otg_08:41:212
3225199243,5cyclictest8050irq/56-dwc_otg_08:07:411
3225199242,4cyclictest0-21swapper/108:26:011
3225199241,5cyclictest0-21swapper/108:30:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional