You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 08:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sun Dec 14, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30776991057,87cyclictest0-21swapper/022:53:020
30776991057,87cyclictest0-21swapper/022:53:020
207622970,6sleep2301ktimersoftd/220:13:052
15412820,2sleep10-21swapper/119:18:141
33142790,2sleep20-21swapper/220:53:102
3077699796,49cyclictest0-21swapper/020:42:590
120022790,7sleep03077699cyclictest21:18:080
30776997720,5cyclictest16764-21grep00:28:160
30776997719,10cyclictest25639-21latency_hist21:55:030
3077699737,14cyclictest0-21swapper/023:52:580
3077699691,24cyclictest0-21swapper/021:28:100
3077699691,21cyclictest0-21swapper/022:38:010
68172670,2sleep10-21swapper/121:03:111
3077699672,28cyclictest0-21swapper/020:53:130
3077699661,29cyclictest0-21swapper/023:28:000
180442650,3sleep218046-21grep21:33:152
3077699637,14cyclictest0-21swapper/020:18:010
3077699637,14cyclictest0-21swapper/020:18:010
30777996019,26cyclictest0-21swapper/123:58:011
30776996030,17cyclictest2725-21latency_hist23:50:030
3046725922,30sleep00-21swapper/019:09:330
3077699571,33cyclictest2016-21cat19:20:020
3077999565,10cyclictest0-21swapper/321:42:583
3077699565,17cyclictest7950irq/56-dwc_otg_23:58:010
30776995627,13cyclictest0-21swapper/021:43:140
30776995610,5cyclictest0-21swapper/022:25:050
3077699551,43cyclictest14331-21ls00:23:080
3077699551,43cyclictest14331-21ls00:23:080
30776995428,6cyclictest17950-21grep20:03:170
3077699542,36cyclictest0-21swapper/022:15:580
3077699542,36cyclictest0-21swapper/022:15:580
3077699542,28cyclictest0-21swapper/023:38:110
3077699542,28cyclictest0-21swapper/000:33:020
3077699539,24cyclictest10494-21fschecks_count00:13:060
30776995340,3cyclictest0-21swapper/000:40:020
30776995332,6cyclictest7750irq/56-dwc_otg23:08:080
30776995330,6cyclictest17068-21wget21:33:020
30776995329,14cyclictest0-21swapper/022:48:150
30776995328,3cyclictest0-21swapper/020:50:020
30776995328,15cyclictest0-21swapper/022:45:020
30776995326,5cyclictest26006-21wget20:28:000
3077699532,34cyclictest0-21swapper/019:13:050
3077699532,27cyclictest0-21swapper/020:00:020
30776995311,25cyclictest13266-21aten_rbpower_en19:53:010
3077799523,9cyclictest0-21swapper/121:47:571
3077799514,12cyclictest0-21swapper/121:52:591
30776995128,5cyclictest2410-21aten_rbpower_vo19:23:000
30776995121,6cyclictest9450irq/48-DMA09:42:410
30776995118,5cyclictest24412-21fschecks_count20:23:050
3077699511,29cyclictest30422-21switchtime20:38:130
3077699511,26cyclictest4616-21fschecks_time20:58:050
30779995029,5cyclictest39-21ksoftirqd/323:58:153
30776995033,4cyclictest7850irq/56-dwc_otg_22:12:560
30776995023,7cyclictest32158-21fschecks_time23:43:050
3077699502,26cyclictest0-21swapper/021:48:040
30776995021,5cyclictest7850irq/56-dwc_otg_23:35:010
30776995021,5cyclictest7850irq/56-dwc_otg_23:35:010
3077699501,24cyclictest18386-21tail23:03:160
3077799492,7cyclictest7950irq/56-dwc_otg_20:42:591
30776994926,5cyclictest0-21swapper/000:18:000
30776994923,15cyclictest0-21swapper/020:08:070
3077699491,26cyclictest10228-21cat21:13:070
117162490,1sleep20-21swapper/200:15:032
30776994831,6cyclictest7950irq/56-dwc_otg_23:23:120
30776994829,5cyclictest7750irq/56-dwc_otg19:48:140
3077799472,4cyclictest0-21swapper/100:13:021
30776994725,3cyclictest0-21swapper/022:03:070
30776994714,26cyclictest0-21swapper/023:14:020
3077899467,25cyclictest0-21swapper/222:15:082
3077899467,25cyclictest0-21swapper/222:15:082
3077699462,4cyclictest0-21swapper/000:05:030
30776994624,6cyclictest7850irq/56-dwc_otg_21:38:120
30776994610,34cyclictest7518-21latency_hist21:05:030
3077999457,23cyclictest0-21swapper/322:20:083
3077799457,25cyclictest0-21swapper/119:58:011
3077799453,8cyclictest0-21swapper/120:13:021
3077799453,14cyclictest7750irq/56-dwc_otg22:47:571
30776994525,5cyclictest8113-21fschecks_time19:38:030
30776994517,13cyclictest0-21swapper/019:31:240
30776994516,15cyclictest8837-21if_wlan000:08:070
30776994510,18cyclictest13-21rcuc/022:33:080
3077999447,25cyclictest0-21swapper/321:53:013
3077999447,22cyclictest0-21swapper/319:35:073
3077999444,6cyclictest0-21swapper/300:03:023
3077799447,28cyclictest0-21swapper/120:03:031
30776994424,4cyclictest10948-21idleruntime-cro19:45:020
30776994424,4cyclictest10948-21idleruntime-cro19:45:020
30776994422,3cyclictest0-21swapper/019:29:180
30776994416,13cyclictest0-21swapper/021:05:050
30776994416,13cyclictest0-21swapper/021:05:050
30776994416,12cyclictest1125-21runrttasks23:15:080
3077699441,21cyclictest0-21swapper/020:33:220
3077899437,24cyclictest0-21swapper/221:47:592
3077799437,26cyclictest0-21swapper/121:33:011
3077799437,24cyclictest0-21swapper/123:43:011
30776994323,13cyclictest0-21swapper/021:58:160
3077699431,23cyclictest16652-21grep22:58:140
291682436,26sleep229169-21munin-node19:07:572
3077999427,23cyclictest0-21swapper/322:43:013
3077999422,21cyclictest1780-21ls22:18:133
3077999422,21cyclictest1780-21ls22:18:133
3077799427,25cyclictest0-21swapper/121:18:021
3077799427,23cyclictest0-21swapper/122:20:071
3077799426,26cyclictest0-21swapper/120:58:011
3077999416,26cyclictest0-21swapper/322:58:003
3077999413,7cyclictest7950irq/56-dwc_otg_20:48:003
3077899417,25cyclictest0-21swapper/221:18:022
3077799417,25cyclictest0-21swapper/119:22:591
3077799417,24cyclictest0-21swapper/100:33:041
3077799414,10cyclictest7750irq/56-dwc_otg23:48:011
3077699417,25cyclictest0-21swapper/022:08:020
3077699417,25cyclictest0-21swapper/022:08:020
30776994117,16cyclictest0-21swapper/021:23:530
3077699411,13cyclictest0-21swapper/020:12:120
138072410,3sleep121-21rcuc/119:53:081
3077999407,23cyclictest0-21swapper/300:23:023
3077999407,23cyclictest0-21swapper/300:23:023
3077999402,2cyclictest0-21swapper/320:37:583
3077899407,23cyclictest0-21swapper/221:58:012
3077799407,23cyclictest0-21swapper/123:08:011
3077799401,6cyclictest7950irq/56-dwc_otg_00:38:011
30777994016,13cyclictest0-21swapper/122:37:581
3077999396,8cyclictest7950irq/56-dwc_otg_23:48:013
30778993910,23cyclictest29-21rcuc/220:57:592
30777993919,4cyclictest2784950wget22:03:021
30777993917,5cyclictest5936-21aten_rbpower_cu19:33:011
3077999387,24cyclictest0-21swapper/300:28:013
3077999382,14cyclictest39-21ksoftirqd/320:10:033
30777993819,5cyclictest5422-21/usr/sbin/munin22:28:101
30777993817,4cyclictest14695-21munin-plugin-st21:25:021
3077999376,14cyclictest39-21ksoftirqd/320:45:023
3077999374,15cyclictest7950irq/56-dwc_otg_20:03:043
3077999373,5cyclictest7950irq/56-dwc_otg_21:03:023
3077999371,3cyclictest0-21swapper/319:58:003
3077899372,6cyclictest7750irq/56-dwc_otg21:53:052
3077899371,16cyclictest3630-21modprobe19:25:012
3077799375,21cyclictest16278-21kworker/u8:219:35:081
30777993717,10cyclictest6500-21aten_rbpower_cu22:33:021
3077799371,27cyclictest12356-21cut00:18:051
88612360,8sleep33077999cyclictest21:08:123
88612360,8sleep33077999cyclictest21:08:123
30779993617,4cyclictest1380050wget22:53:033
30779993617,4cyclictest1380050wget22:53:033
30779993614,5cyclictest1125-21runrttasks19:28:033
3077899367,25cyclictest0-21swapper/223:05:082
3077899364,15cyclictest7850irq/56-dwc_otg_19:13:162
30778993616,6cyclictest6616-21apt-get00:03:012
3077799363,27cyclictest7950irq/56-dwc_otg_23:17:581
3077799361,6cyclictest0-21swapper/122:57:591
3077999356,9cyclictest7950irq/56-dwc_otg_23:10:033
3077999356,13cyclictest7950irq/56-dwc_otg_21:18:163
3077999355,5cyclictest7950irq/56-dwc_otg_23:33:003
3077999355,5cyclictest7950irq/56-dwc_otg_23:33:003
3077999354,10cyclictest7750irq/56-dwc_otg23:45:023
3077999353,18cyclictest11942-21grep22:48:003
3077999351,13cyclictest7750irq/56-dwc_otg00:18:043
30778993518,4cyclictest1225-21aten_rbpower_cu23:48:032
3077899351,3cyclictest0-21swapper/221:38:022
3077899351,24cyclictest0-21swapper/221:23:002
3077799353,4cyclictest7950irq/56-dwc_otg_22:08:011
3077799353,4cyclictest7950irq/56-dwc_otg_22:08:011
3077999342,21cyclictest0-21swapper/320:30:563
3077999342,14cyclictest7950irq/56-dwc_otg_23:13:163
30779993416,4cyclictest2284-21/usr/sbin/munin19:23:113
3077899343,18cyclictest4702-21aten_rbpower_en22:27:592
3077899341,8cyclictest5911-21tail23:58:162
3077899341,23cyclictest17795-21timerwakeupswit20:03:162
3077799349,11cyclictest7950irq/56-dwc_otg_00:25:021
3077799349,11cyclictest7950irq/56-dwc_otg_00:25:021
3077799348,7cyclictest0-21swapper/120:47:591
3077799347,17cyclictest0-21swapper/122:53:021
3077799347,17cyclictest0-21swapper/122:53:021
3077799347,14cyclictest0-21swapper/122:15:081
3077799347,14cyclictest0-21swapper/122:15:081
3077799342,18cyclictest0-21swapper/119:48:071
3077999337,4cyclictest7750irq/56-dwc_otg22:02:593
3077999335,14cyclictest7950irq/56-dwc_otg_23:53:013
3077999332,5cyclictest11738-21latency_hist00:15:033
3077899336,13cyclictest7950irq/56-dwc_otg_22:58:052
3077899335,14cyclictest7950irq/56-dwc_otg_22:12:572
3077899333,5cyclictest13369-21cpuspeed19:53:022
30778993313,6cyclictest29941-21if_err_wlan020:38:062
3077899331,19cyclictest27396-21ls20:30:022
3077799337,5cyclictest17723-21if_wlan023:03:071
3077799331,3cyclictest7950irq/56-dwc_otg_20:08:001
30777993313,5cyclictest435-21cron23:30:021
3077799331,22cyclictest19083-21cut21:38:071
3077999329,5cyclictest8517-21/usr/sbin/munin00:08:023
3077999327,14cyclictest0-21swapper/321:28:023
3077999326,7cyclictest7950irq/56-dwc_otg_23:03:023
3077999324,5cyclictest7750irq/56-dwc_otg19:43:083
3077999324,5cyclictest7750irq/56-dwc_otg19:43:083
3077999322,22cyclictest0-21swapper/320:16:143
3077999322,22cyclictest0-21swapper/320:16:143
30779993217,6cyclictest26199-21fschecks_count20:28:033
30779993210,14cyclictest39-21ksoftirqd/322:12:563
3077899327,20cyclictest0-21swapper/219:48:012
3077899327,13cyclictest7950irq/56-dwc_otg_22:08:062
3077899327,13cyclictest7950irq/56-dwc_otg_22:08:062
3077899325,6cyclictest7850irq/56-dwc_otg_21:03:062
3077899325,25cyclictest0-21swapper/223:23:012
3077899323,5cyclictest0-21swapper/223:43:132
3077899323,16cyclictest28224-21apt-get23:33:002
3077899323,16cyclictest28224-21apt-get23:33:002
3077899322,18cyclictest0-21swapper/220:09:212
3077899321,4cyclictest0-21swapper/223:52:582
3077899321,15cyclictest1377350wget22:53:032
3077899321,15cyclictest1377350wget22:53:032
3077799328,5cyclictest7750irq/56-dwc_otg20:20:011
3077799328,5cyclictest7750irq/56-dwc_otg20:20:011
3077799323,5cyclictest21792-21processes23:13:131
3077799322,20cyclictest0-21swapper/123:34:321
3077799322,20cyclictest0-21swapper/123:34:321
3077799321,28cyclictest31705-21grep22:13:031
3077799321,27cyclictest8266-21ls21:08:041
3077799321,27cyclictest8266-21ls21:08:041
3077799321,19cyclictest0-21swapper/123:23:151
3077799321,18cyclictest29633-21aten_rbpower_po20:38:021
3077799321,15cyclictest27338-21ls22:00:021
3077999317,22cyclictest0-21swapper/300:33:043
3077999316,6cyclictest7950irq/56-dwc_otg_22:08:163
3077999316,6cyclictest7950irq/56-dwc_otg_22:08:163
3077999316,15cyclictest0-21swapper/322:25:073
3077999314,4cyclictest7850irq/56-dwc_otg_19:13:033
3077999313,18cyclictest0-21swapper/300:35:593
3077999312,5cyclictest39-21ksoftirqd/322:38:073
3077999312,20cyclictest0-21swapper/323:26:403
3077999311,20cyclictest11603-21cpuspeed19:48:033
3077999311,16cyclictest25591-21ls20:25:023
3077999311,11cyclictest7950irq/56-dwc_otg_21:23:073
3077899318,10cyclictest7950irq/56-dwc_otg_00:38:012
3077899312,21cyclictest0-21swapper/221:42:182
30778993120,7cyclictest18620-21cron23:05:022
30778993118,4cyclictest15828-21cpuspeed00:28:032
3077799316,10cyclictest7950irq/56-dwc_otg_20:28:051
3077799315,6cyclictest3031-21latency20:53:051
3077799313,4cyclictest0-21swapper/123:38:151
3077799312,19cyclictest0-21swapper/100:01:231
3077799312,15cyclictest24188-21aten_rbpower_en20:23:021
3077799311,8cyclictest16688-21grep00:28:151
30777993117,4cyclictest83922chrt00:07:491
30777993113,4cyclictest29239-21latency_hist20:35:021
30777993112,4cyclictest20843-21fschecks_time21:43:041
3077999306,22cyclictest0-21swapper/322:33:023
3077999305,5cyclictest37-21rcuc/320:53:003
3077999304,16cyclictest1125-21runrttasks21:47:483
3077999303,13cyclictest7950irq/56-dwc_otg_19:20:023
30779993020,8cyclictest11-21rcu_preempt19:52:593
30779993014,8cyclictest37-21rcuc/320:12:593
3077899307,4cyclictest7950irq/56-dwc_otg_23:13:082
3077899306,6cyclictest7850irq/56-dwc_otg_00:08:162
3077899305,6cyclictest9335-21cat21:10:032
3077899305,6cyclictest9335-21cat21:10:032
3077899304,6cyclictest7950irq/56-dwc_otg_19:43:032
3077899304,6cyclictest7950irq/56-dwc_otg_19:43:032
3077899304,6cyclictest7850irq/56-dwc_otg_19:37:592
3077899302,4cyclictest0-21swapper/219:28:032
3077899301,5cyclictest1367-21grep20:48:092
3077899301,18cyclictest16002-21ls21:28:142
3077899301,14cyclictest7750irq/56-dwc_otg22:01:392
30778993010,4cyclictest0-21swapper/200:20:022
3077799303,7cyclictest7750irq/56-dwc_otg21:28:161
30777993010,4cyclictest5444-21apt19:30:021
299822307,11sleep10-21swapper/119:08:121
3077999294,17cyclictest39-21ksoftirqd/319:33:013
3077999292,6cyclictest7750irq/56-dwc_otg23:40:023
3077999292,5cyclictest0-21swapper/321:58:013
3077999292,19cyclictest0-21swapper/323:15:223
3077999291,8cyclictest11252-21latency_hist21:15:023
3077899296,14cyclictest0-21swapper/219:15:082
3077899295,5cyclictest7950irq/56-dwc_otg_20:17:582
3077899295,5cyclictest7950irq/56-dwc_otg_20:17:582
3077899295,4cyclictest7850irq/56-dwc_otg_20:23:072
30778992922,5cyclictest7498-21latency_hist19:35:032
3077899292,19cyclictest0-21swapper/220:30:502
30778992921,5cyclictest15178-21idleruntime-cro00:25:022
30778992921,5cyclictest15178-21idleruntime-cro00:25:022
3077899291,25cyclictest10436-21ls22:43:052
3077899291,19cyclictest0-21swapper/221:11:082
3077899291,13cyclictest7750irq/56-dwc_otg19:58:042
30777992913,6cyclictest1125-21runrttasks19:41:161
30777992913,6cyclictest1125-21runrttasks19:41:161
3077999281,19cyclictest0-21swapper/321:30:043
3077899282,14cyclictest0-21swapper/222:36:232
3077899281,6cyclictest3334-21missed_timers22:23:102
30778992812,4cyclictest30012-21/usr/sbin/munin23:38:092
3077899281,16cyclictest0-21swapper/222:45:422
3077799281,5cyclictest11218-21unixbench_singl22:43:151
3077799281,14cyclictest7950irq/56-dwc_otg_21:11:251
3077799281,14cyclictest0-21swapper/119:12:381
3077799281,13cyclictest7950irq/56-dwc_otg_23:53:111
30779992719,6cyclictest24453-21sshd23:21:063
3077899277,4cyclictest0-21swapper/223:18:062
3077899271,4cyclictest0-21swapper/200:33:142
3077899271,3cyclictest7950irq/56-dwc_otg_20:43:012
3077999262,15cyclictest0-21swapper/320:57:173
3077999261,17cyclictest0-21swapper/321:37:023
3077899266,4cyclictest7950irq/56-dwc_otg_23:28:112
3077899262,10cyclictest7850irq/56-dwc_otg_22:33:092
290422263,14sleep30-21swapper/319:06:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional