You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 20:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sun Dec 14, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1575521030,2sleep215756-21/usr/sbin/munin07:18:102
130099910342,15cyclictest0-21swapper/008:25:080
239312780,6sleep01300999cyclictest10:38:160
327012770,2sleep10-21swapper/111:03:161
304152760,6sleep01300999cyclictest09:28:090
13009997511,17cyclictest0-21swapper/009:12:590
1300999706,25cyclictest0-21swapper/011:33:020
1300999706,25cyclictest0-21swapper/011:33:020
13009997056,6cyclictest101ktimersoftd/008:45:020
13012996932,15cyclictest5209-21/usr/sbin/munin08:18:143
13012996834,21cyclictest3124-21irqstats08:13:073
13009996813,9cyclictest0-21swapper/008:22:590
1301199675,3cyclictest7750irq/56-dwc_otg11:00:092
1300999677,41cyclictest0-21swapper/010:58:010
1300999677,41cyclictest0-21swapper/010:58:010
1300999675,34cyclictest0-21swapper/009:03:010
13012996530,19cyclictest30478-21grep12:28:033
13012996527,22cyclictest18811-21taskset08:55:493
13009996516,9cyclictest0-21swapper/007:33:010
13012996428,19cyclictest13540-21wc11:40:023
13009996344,6cyclictest20477-21grep11:58:290
1300999631,33cyclictest0-21swapper/009:18:160
13012996222,19cyclictest7950irq/56-dwc_otg_12:15:033
1300999626,54cyclictest0-21swapper/008:53:010
1300999621,32cyclictest26653-21ls10:48:010
14502610,1sleep20-21swapper/208:08:092
13012996133,13cyclictest15994-21entropy11:48:073
13012996131,17cyclictest30288-21/usr/sbin/munin09:28:073
13012996129,18cyclictest28206-21awk09:23:033
1301099612,5cyclictest0-21swapper/112:32:591
13009996113,34cyclictest0-21swapper/007:20:020
13009996035,12cyclictest28041-21cron12:20:020
13009996035,12cyclictest28041-21cron12:20:020
1300999602,36cyclictest0-21swapper/007:48:010
1300999602,36cyclictest0-21swapper/007:48:010
13012995933,7cyclictest3827-21ntp_states11:13:123
13012995928,20cyclictest9998-21munin-node10:02:583
13012995925,17cyclictest7950irq/56-dwc_otg_12:23:063
1301199594,7cyclictest0-21swapper/210:30:032
1301199592,4cyclictest0-21swapper/211:33:062
1301199592,4cyclictest0-21swapper/211:33:062
1300999599,30cyclictest3765-21unixbench_multi08:13:160
13012995834,8cyclictest20384-21apt-get10:30:093
13012995831,13cyclictest15879-21/usr/sbin/munin08:48:073
13012995831,13cyclictest15879-21/usr/sbin/munin08:48:073
13012995831,12cyclictest29598-21apt-get07:58:013
13012995830,13cyclictest424-21latency_hist08:05:033
1301199582,12cyclictest7950irq/56-dwc_otg_07:38:052
13010995811,22cyclictest0-21swapper/111:57:591
1300999584,27cyclictest0-21swapper/009:52:590
13009995820,20cyclictest0-21swapper/012:28:000
13012995722,18cyclictest7950irq/56-dwc_otg_09:58:023
1301199571,19cyclictest0-21swapper/210:33:022
13009995733,13cyclictest15316-21sh11:45:020
1300999571,37cyclictest17662-21aten_rbpower_po11:53:020
13012995629,16cyclictest1124-21snmpd09:06:353
13012995629,13cyclictest5520-21/usr/sbin/munin11:18:153
13012995620,10cyclictest7950irq/56-dwc_otg_07:38:023
13009995642,4cyclictest5227-21grep08:18:140
13009995628,6cyclictest7950irq/56-dwc_otg_09:00:020
1300999562,30cyclictest0-21swapper/007:13:040
1300999561,51cyclictest101ktimersoftd/010:08:150
50912550,2sleep10-21swapper/108:18:101
13012995528,11cyclictest29039-21taskset07:54:003
13012995525,11cyclictest25822-21uptime10:43:163
13012995522,10cyclictest7950irq/56-dwc_otg_09:15:033
13009995536,4cyclictest5529-21missed_timers11:18:150
1300999553,27cyclictest0-21swapper/011:38:000
13012995430,12cyclictest24171-21/usr/sbin/munin07:43:183
13012995430,12cyclictest24171-21/usr/sbin/munin07:43:183
13012995427,15cyclictest17852-21/usr/sbin/munin08:53:093
13012995427,11cyclictest18241-21/usr/sbin/munin11:53:103
13012995420,19cyclictest11614-21runrttasks10:05:403
13012995419,11cyclictest7950irq/56-dwc_otg_08:28:053
1300999546,28cyclictest9-21ksoftirqd/011:10:030
1300999544,47cyclictest9-21ksoftirqd/008:10:020
13009995424,5cyclictest25336-21grep07:43:160
13009995424,5cyclictest25336-21grep07:43:160
1300999542,30cyclictest4250-21grep11:13:160
1300999542,27cyclictest26245-21apt-config12:15:030
1300999542,26cyclictest0-21swapper/011:25:030
13009995421,15cyclictest17075-21/usr/sbin/munin07:23:030
13012995341,8cyclictest13253-21apt-get07:12:593
13012995327,12cyclictest27584-21ls09:20:023
13012995324,11cyclictest22439-21chrt12:04:123
13012995324,10cyclictest22829-21/usr/sbin/munin10:38:133
13012995322,12cyclictest27215-21chrt07:48:383
13012995322,12cyclictest27215-21chrt07:48:383
13012995322,10cyclictest14838-21taskset10:13:433
1301199537,43cyclictest20008-21timerwakeupswit08:58:162
1300999533,29cyclictest0-21swapper/012:23:060
13009995330,16cyclictest0-21swapper/009:10:030
1300999531,7cyclictest18997-21fschecks_time07:28:050
1300999531,28cyclictest10501-21grep08:33:080
13012995223,14cyclictest20641-21taskset09:01:113
13012995220,12cyclictest28040-21cron12:20:023
13012995220,12cyclictest28040-21cron12:20:023
1300999529,23cyclictest0-21swapper/010:05:030
1300999522,27cyclictest0-21swapper/012:38:100
1300999521,24cyclictest0-21swapper/008:47:590
1300999521,24cyclictest0-21swapper/008:47:590
13012995123,15cyclictest20303-21grep11:58:273
13012995123,15cyclictest0-21swapper/307:22:093
13012995123,11cyclictest4509-21taskset09:46:513
13012995120,14cyclictest3731-21cat09:43:163
13012995118,10cyclictest7950irq/56-dwc_otg_11:31:453
13012995118,10cyclictest7950irq/56-dwc_otg_11:31:453
1301099514,19cyclictest7850irq/56-dwc_otg_12:38:021
1300999512,29cyclictest0-21swapper/010:28:080
1300999511,7cyclictest28277-21fschecks_count09:23:050
1300999511,16cyclictest0-21swapper/009:43:170
13009995111,22cyclictest0-21swapper/010:43:010
13012995025,12cyclictest15399-21sed11:45:023
13012995021,10cyclictest7950irq/56-dwc_otg_12:31:103
13012995020,11cyclictest607-21chrt08:05:043
13012995019,15cyclictest7950irq/56-dwc_otg_09:52:233
13012995018,16cyclictest7950irq/56-dwc_otg_09:33:503
13012995018,11cyclictest7850irq/56-dwc_otg_09:39:143
13012995018,11cyclictest7850irq/56-dwc_otg_09:39:143
13010995012,8cyclictest0-21swapper/108:57:591
1300999504,27cyclictest0-21swapper/011:48:130
13009995019,17cyclictest0-21swapper/010:30:080
13009995018,17cyclictest971-21/usr/sbin/munin09:38:080
13009995018,17cyclictest971-21/usr/sbin/munin09:38:080
13012994935,11cyclictest10960-21ls08:33:153
13012994923,9cyclictest7950irq/56-dwc_otg_12:06:593
13012994922,11cyclictest20461-21taskset07:32:513
1301099497,10cyclictest7850irq/56-dwc_otg_07:12:581
13009994923,2cyclictest0-21swapper/008:38:000
13009994921,7cyclictest32255-21fschecks_time11:03:060
13009994921,16cyclictest0-21swapper/012:03:160
13009994914,18cyclictest8525-21fschecks_time09:58:050
1300999491,26cyclictest13937-21fschecks_time10:13:060
13012994830,15cyclictest9417-21sed11:28:123
13012994823,10cyclictest0-21swapper/308:42:413
13009994824,6cyclictest7850irq/56-dwc_otg_10:55:020
13009994821,14cyclictest7950irq/56-dwc_otg_07:58:150
13009994818,14cyclictest31768-21fschecks_count08:03:050
1300999481,27cyclictest0-21swapper/012:08:040
13012994731,12cyclictest1455-21/usr/sbin/munin12:38:143
13012994720,11cyclictest7850irq/56-dwc_otg_11:21:183
13012994719,10cyclictest7950irq/56-dwc_otg_10:48:443
1300999475,29cyclictest9-21ksoftirqd/007:38:070
13009994722,7cyclictest7850irq/56-dwc_otg_07:55:020
13012994632,11cyclictest19399-21/usr/sbin/munin10:28:053
13012994631,12cyclictest19737-21grep07:28:153
13012994631,11cyclictest16584-21cat07:20:033
13012994629,13cyclictest29351-21grep10:53:143
13012994629,13cyclictest16758-21cron10:20:023
13012994622,9cyclictest7950irq/56-dwc_otg_08:37:253
13012994622,12cyclictest7950irq/56-dwc_otg_10:21:523
1301199467,25cyclictest0-21swapper/207:40:082
1301199467,25cyclictest0-21swapper/207:40:082
1301199465,15cyclictest0-21swapper/211:38:012
13012994529,12cyclictest7146-21cat08:23:133
13012994521,15cyclictest7950irq/56-dwc_otg_11:04:533
1301199457,25cyclictest0-21swapper/208:48:012
1301199457,25cyclictest0-21swapper/208:48:012
1301199455,11cyclictest0-21swapper/210:08:002
13011994524,5cyclictest2885-21cat12:40:022
120002453,7sleep00-21swapper/007:08:100
120002453,7sleep00-21swapper/007:08:100
13012994420,10cyclictest0-21swapper/311:07:283
1301199443,7cyclictest0-21swapper/209:22:592
1300999448,20cyclictest0-21swapper/010:20:370
57982430,7sleep11301099cyclictest09:48:181
13012994328,12cyclictest30094-21aten_rbpower_cu10:58:013
13012994328,12cyclictest30094-21aten_rbpower_cu10:58:013
13011994317,5cyclictest7766-21idleruntime-cro09:55:022
1300999432,8cyclictest16760-21cron10:20:020
13009994319,14cyclictest1125-21runrttasks11:27:480
13009994319,10cyclictest0-21swapper/009:33:090
1301099423,7cyclictest0-21swapper/111:43:001
1301199417,25cyclictest0-21swapper/209:13:012
1301099417,25cyclictest0-21swapper/112:28:021
1301099412,11cyclictest7750irq/56-dwc_otg11:27:591
13010994119,5cyclictest2987-21fschecks_time08:13:061
13009994112,3cyclictest0-21swapper/009:46:430
1301099401,9cyclictest7850irq/56-dwc_otg_10:27:581
13011993929,8cyclictest14081-21uname10:13:082
1301099392,6cyclictest7850irq/56-dwc_otg_11:33:001
1301099392,6cyclictest7850irq/56-dwc_otg_11:33:001
1301099391,7cyclictest7950irq/56-dwc_otg_08:52:591
1301199387,24cyclictest0-21swapper/212:33:012
1301199387,22cyclictest0-21swapper/208:38:012
1301099386,22cyclictest10945-21kworker/u8:008:00:081
1301099381,3cyclictest0-21swapper/107:53:011
13010993812,5cyclictest0-21swapper/112:23:141
1300999382,26cyclictest0-21swapper/012:31:000
13011993718,6cyclictest17452-21seq11:51:372
1301099375,25cyclictest7850irq/56-dwc_otg_09:12:591
1301099372,6cyclictest7950irq/56-dwc_otg_12:17:581
1301099372,6cyclictest7950irq/56-dwc_otg_12:17:581
13010993716,6cyclictest20019-21sh07:30:021
13010993712,7cyclictest14942-21cron10:15:021
13011993630,4cyclictest301ktimersoftd/211:22:592
1301099368,10cyclictest0-21swapper/108:08:011
1301099366,17cyclictest7950irq/56-dwc_otg_10:03:161
13010993631,3cyclictest21-21rcuc/112:13:001
1301099362,16cyclictest30119-21fschecks_time09:28:051
1301099361,24cyclictest12654-21switchtime08:38:141
1301199353,14cyclictest21172-21/usr/sbin/munin12:03:142
1301199352,14cyclictest7850irq/56-dwc_otg_11:28:042
1301199351,24cyclictest5302-21taskset11:18:052
1301099352,9cyclictest7750irq/56-dwc_otg09:32:591
13010993518,4cyclictest26213-21/usr/sbin/munin07:48:051
13010993518,4cyclictest26213-21/usr/sbin/munin07:48:051
1301199341,5cyclictest27100-21unixbench_multi07:48:172
1301199341,5cyclictest27100-21unixbench_multi07:48:172
1301199341,24cyclictest17751-21latency10:23:082
1301099348,5cyclictest7850irq/56-dwc_otg_11:55:021
1301099347,16cyclictest8264-21wget08:28:021
1301099346,6cyclictest7850irq/56-dwc_otg_10:53:141
1301099343,10cyclictest0-21swapper/109:10:021
1301099342,5cyclictest24328-21apt12:10:021
1301099342,3cyclictest7850irq/56-dwc_otg_07:43:001
1301099342,3cyclictest7850irq/56-dwc_otg_07:43:001
1301099342,18cyclictest0-21swapper/110:35:031
13010993419,4cyclictest1125-21runrttasks07:18:031
1301199333,7cyclictest7850irq/56-dwc_otg_11:43:032
1301199332,16cyclictest31-21ksoftirqd/209:48:152
1301199332,13cyclictest7850irq/56-dwc_otg_10:40:032
1301199331,6cyclictest2454-21latency_hist09:40:032
1301199331,6cyclictest2454-21latency_hist09:40:032
13011993313,6cyclictest4814-21fschecks_time08:18:062
1301199331,22cyclictest3021-21idleruntime08:13:062
1301099338,5cyclictest23-21ksoftirqd/109:53:101
1301099336,5cyclictest7750irq/56-dwc_otg12:03:041
1301099334,7cyclictest7850irq/56-dwc_otg_10:00:021
1301099331,28cyclictest17194-21cut07:23:061
1301099331,22cyclictest6562-21cut08:23:041
1301199325,25cyclictest0-21swapper/210:53:002
1301199323,15cyclictest7950irq/56-dwc_otg_12:28:162
13011993226,4cyclictest21822-21cron07:35:022
13011993223,3cyclictest7850irq/56-dwc_otg_08:53:062
1301199321,5cyclictest10686-21ntp_kernel_pll_10:03:102
1301199321,29cyclictest17795-21ls07:23:142
1301099327,5cyclictest7950irq/56-dwc_otg_09:22:591
1301099327,5cyclictest12104-21fschecks_time10:08:051
1301099325,7cyclictest7850irq/56-dwc_otg_09:38:001
1301099325,7cyclictest7850irq/56-dwc_otg_09:38:001
1301099325,4cyclictest7750irq/56-dwc_otg11:13:051
1301099324,8cyclictest7750irq/56-dwc_otg10:48:061
1301099322,5cyclictest26438-21df09:18:041
1301099322,19cyclictest0-21swapper/108:47:491
1301099322,19cyclictest0-21swapper/108:47:491
1301099321,8cyclictest16581-21sed10:18:161
1301099321,29cyclictest13909-21cut08:43:061
1301199313,5cyclictest7950irq/56-dwc_otg_09:43:172
1301199312,6cyclictest11328-21munin-run08:35:022
13011993114,4cyclictest2766-21latency_hist11:10:032
1301199311,14cyclictest7750irq/56-dwc_otg09:58:072
1301099316,7cyclictest7750irq/56-dwc_otg10:58:021
1301099316,7cyclictest7750irq/56-dwc_otg10:58:021
1301099313,14cyclictest7950irq/56-dwc_otg_11:18:161
1301099312,8cyclictest7850irq/56-dwc_otg_11:08:181
13010993116,12cyclictest0-21swapper/107:38:151
1301199302,6cyclictest25788-21grep10:43:152
13011993021,7cyclictest29128-21cron07:55:022
1301199301,15cyclictest31-21ksoftirqd/211:58:042
1301099306,15cyclictest0-21swapper/110:35:081
1301099303,14cyclictest7950irq/56-dwc_otg_07:33:081
1301099301,7cyclictest18391-21unixbench_singl10:23:161
1301099301,14cyclictest0-21swapper/109:45:021
1301199297,14cyclictest0-21swapper/208:00:072
1301199297,10cyclictest7750irq/56-dwc_otg10:16:502
1301199295,14cyclictest0-21swapper/208:43:032
1301199293,17cyclictest0-21swapper/209:05:392
1301199291,6cyclictest7850irq/56-dwc_otg_10:57:592
1301199291,6cyclictest7850irq/56-dwc_otg_10:57:592
1301199291,5cyclictest7394-21grep08:23:152
1301199291,4cyclictest0-21swapper/209:28:132
13011992912,4cyclictest29833-21cron12:25:022
13011992910,4cyclictest2303850wget12:08:042
1301099299,13cyclictest7090-21perl11:23:051
1301099295,5cyclictest29682-21wget07:58:011
13010992911,13cyclictest7850irq/56-dwc_otg_09:03:011
112312293,15sleep10-21swapper/107:05:491
112312293,15sleep10-21swapper/107:05:491
13011992822,4cyclictest20023-21idleruntime-cro07:30:022
1301199282,17cyclictest0-21swapper/211:11:442
1301199282,16cyclictest7850irq/56-dwc_otg_12:11:342
13011992817,8cyclictest15850-21aten_rbpower_po11:48:052
1301199281,6cyclictest26879-21/usr/sbin/munin12:18:012
1301199281,6cyclictest26879-21/usr/sbin/munin12:18:012
1301199281,5cyclictest7850irq/56-dwc_otg_08:28:052
1301199281,10cyclictest0-21swapper/209:17:472
1301099283,5cyclictest7750irq/56-dwc_otg10:43:011
1301099282,6cyclictest11108-21unixbench_singl08:33:161
1301199271,4cyclictest7750irq/56-dwc_otg09:03:002
1301199271,18cyclictest0-21swapper/207:10:402
1301199271,14cyclictest26624-21ls10:48:012
1301199271,12cyclictest7950irq/56-dwc_otg_09:32:192
13010992715,4cyclictest1125-21runrttasks11:46:231
1301099271,4cyclictest12157-21/usr/sbin/munin11:38:121
1301199265,5cyclictest29-21rcuc/207:58:042
113722263,14sleep20-21swapper/207:07:402
113722263,14sleep20-21swapper/207:07:402
113062263,14sleep30-21swapper/307:06:483
113062263,14sleep30-21swapper/307:06:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional