You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 17:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Apr 24, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2172824280,5sleep10-21swapper/116:26:061
3402920,4sleep2339-21forks20:26:052
2322599886,50cyclictest0-21swapper/020:26:020
2322599817,65cyclictest0-21swapper/021:51:010
249552790,6sleep22322799cyclictest18:57:252
12362770,2sleep229-21rcuc/221:46:012
173162760,2sleep00-21swapper/019:27:340
160042730,1sleep30-21swapper/320:45:113
2322599719,39cyclictest0-21swapper/016:30:490
23225997148,10cyclictest101ktimersoftd/016:55:480
2322599692,33cyclictest12871-21tune2fs20:41:040
2322899671,10cyclictest0-21swapper/316:36:003
23225996745,10cyclictest101ktimersoftd/017:21:140
99422660,5sleep22322799cyclictest19:57:472
2322699665,22cyclictest7750irq/56-dwc_otg21:16:011
23225996629,6cyclictest0-21swapper/018:36:000
2322899652,14cyclictest0-21swapper/316:50:593
2322599641,24cyclictest0-21swapper/018:21:010
2322899633,11cyclictest11932-21ssh20:00:593
23225996340,11cyclictest101ktimersoftd/020:18:160
2322599624,54cyclictest101ktimersoftd/019:06:160
23225996234,13cyclictest101ktimersoftd/021:26:110
23225996224,19cyclictest0-21swapper/019:50:590
23225996212,33cyclictest0-21swapper/020:31:150
2322699617,21cyclictest0-21swapper/117:11:001
2322599615,49cyclictest0-21swapper/020:56:060
2322599614,39cyclictest27150-21cut18:10:590
23225996137,5cyclictest7750irq/56-dwc_otg19:01:160
23225996117,32cyclictest9997-21timerandwakeup21:16:150
2322599611,46cyclictest101ktimersoftd/018:40:580
23225996042,9cyclictest101ktimersoftd/021:50:210
23225996035,13cyclictest101ktimersoftd/017:00:480
23225996011,34cyclictest101ktimersoftd/017:05:490
23226995928,9cyclictest26880-21apt-get16:41:021
23225995937,12cyclictest7750irq/56-dwc_otg19:15:490
23225995937,12cyclictest7750irq/56-dwc_otg19:15:490
2322599591,48cyclictest21653-21proc_pri20:51:120
2322599591,36cyclictest0-21swapper/019:25:490
2322899584,29cyclictest10200-21kworker/u8:019:45:583
2322599581,5cyclictest30768-21grep21:41:170
2322599581,46cyclictest21566-21latency_hist17:55:490
23226995713,12cyclictest16099-21ssh19:26:121
23226995712,15cyclictest31264-21ssh19:05:591
23225995739,9cyclictest101ktimersoftd/017:16:000
2322599572,8cyclictest0-21swapper/019:45:590
2322599572,45cyclictest0-21swapper/019:41:140
23225995717,25cyclictest0-21swapper/018:06:000
2322599571,41cyclictest10814-21grep21:56:170
2322599564,41cyclictest0-21swapper/019:35:490
23225995635,9cyclictest12328-21idleruntime20:01:060
23225995635,10cyclictest101ktimersoftd/020:06:030
2322599562,30cyclictest23378-21latency_hist18:00:490
23225995613,36cyclictest0-21swapper/018:45:590
2322599561,20cyclictest101ktimersoftd/017:26:150
23226995514,8cyclictest7950irq/56-dwc_otg_21:01:011
23226995512,21cyclictest7850irq/56-dwc_otg_19:01:101
2322599555,35cyclictest0-21swapper/021:06:010
2322599554,47cyclictest0-21swapper/019:56:070
2322599551,32cyclictest26669-21unixbench_multi21:36:160
2322699542,4cyclictest0-21swapper/120:31:001
23225995436,9cyclictest101ktimersoftd/018:57:530
2322599542,32cyclictest24060-21latency_hist20:15:490
23225995419,6cyclictest7750irq/56-dwc_otg21:31:020
287982530,2sleep10-21swapper/118:15:491
2322799536,18cyclictest0-21swapper/221:51:012
23226995311,21cyclictest24679-21sh21:35:041
2322599537,35cyclictest0-21swapper/018:51:000
2322599536,34cyclictest15933-21cron17:40:480
23225995334,6cyclictest7850irq/56-dwc_otg_19:20:490
23225995327,14cyclictest0-21swapper/018:32:350
2322599531,35cyclictest0-21swapper/017:30:590
53552520,2sleep20-21swapper/218:34:462
2322699526,5cyclictest0-21swapper/121:46:001
23226995218,11cyclictest3174-21diskstats19:11:041
23226995218,11cyclictest3174-21diskstats19:11:041
23226995215,20cyclictest13071-21/usr/sbin/munin20:41:061
23226995214,17cyclictest22292-21ssh18:55:071
23225995228,6cyclictest7950irq/56-dwc_otg_16:46:110
2322599521,25cyclictest0-21swapper/021:15:480
23226995118,17cyclictest5723-21ssh18:35:251
23225995121,17cyclictest0-21swapper/016:36:130
2322599511,22cyclictest101ktimersoftd/017:10:490
2322599511,22cyclictest101ktimersoftd/017:10:490
23228995036,6cyclictest381ktimersoftd/321:50:593
23226995017,12cyclictest2232-21missed_timers17:01:101
23226995014,20cyclictest30647-21cat16:51:031
2322599506,34cyclictest0-21swapper/019:39:200
2322599502,33cyclictest0-21swapper/016:41:100
2322599501,5cyclictest18017-21aten_rbpower_cu17:46:010
2322599501,5cyclictest17060-21grep20:46:050
23225995010,37cyclictest101ktimersoftd/021:21:000
23225995010,37cyclictest101ktimersoftd/021:21:000
23228994921,12cyclictest7950irq/56-dwc_otg_19:41:023
2322799493,11cyclictest0-21swapper/217:40:592
2322699499,7cyclictest0-21swapper/118:15:581
2322699497,27cyclictest0-21swapper/119:51:001
23226994910,15cyclictest1652-21diskmemload21:43:031
2322599498,32cyclictest503-21in:imuxsock18:29:400
23225994930,7cyclictest101ktimersoftd/020:21:120
23225994927,10cyclictest101ktimersoftd/021:01:170
227822493,14sleep00-21swapper/016:28:330
2322799483,7cyclictest7950irq/56-dwc_otg_21:16:012
2322699487,26cyclictest0-21swapper/117:26:021
23226994815,19cyclictest18331-21taskset20:46:521
23226994814,19cyclictest9227-21ssh19:17:501
23226994813,19cyclictest28909-21fschecks_count16:46:041
23226994811,17cyclictest7750irq/56-dwc_otg18:41:071
23225994816,25cyclictest0-21swapper/017:11:000
2322699479,22cyclictest1681-21ssh20:26:461
2322699477,26cyclictest0-21swapper/117:31:011
2322699475,23cyclictest31905-21rm19:46:001
23226994714,19cyclictest15711-21chrt21:23:451
23226994714,19cyclictest15711-21chrt21:23:451
23226994711,19cyclictest32507-21cat16:56:041
2322599472,26cyclictest0-21swapper/020:40:190
2322799467,25cyclictest0-21swapper/221:11:022
2322799467,25cyclictest0-21swapper/218:16:002
2322799467,25cyclictest0-21swapper/217:16:012
23227994611,6cyclictest31-21ksoftirqd/220:41:042
2322699469,25cyclictest29392-21sed20:21:141
2322699467,13cyclictest224250ssh21:06:421
2322699465,14cyclictest12601-21ssh20:01:091
2322699464,26cyclictest10237-21rm19:58:191
23226994615,11cyclictest23713-21/usr/sbin/munin16:31:071
23226994612,20cyclictest0-21swapper/121:53:041
23226994610,15cyclictest25784-21ssh19:37:461
2322899457,23cyclictest0-21swapper/318:26:023
2322799457,24cyclictest0-21swapper/216:36:022
2322699459,14cyclictest16292-21ssh18:46:381
2322699459,13cyclictest22004-21ssh20:12:121
2322699458,8cyclictest7850irq/56-dwc_otg_21:10:591
23226994512,18cyclictest29100-21ssh21:40:351
23226994512,11cyclictest25315-21apt-get18:06:011
2322899448,6cyclictest7850irq/56-dwc_otg_18:31:123
2322799447,25cyclictest0-21swapper/221:41:022
2322699448,13cyclictest28027-21df_inode19:41:031
2322699447,24cyclictest0-21swapper/116:36:021
2322699447,22cyclictest12943-21rm22:00:031
23226994417,11cyclictest0-21swapper/121:26:001
2322599445,17cyclictest101ktimersoftd/018:16:240
169302440,4sleep316931-21/usr/sbin/munin20:46:033
2322899435,13cyclictest7950irq/56-dwc_otg_20:36:093
2322799436,24cyclictest0-21swapper/217:21:012
23227994310,19cyclictest19654-21ntp_kernel_pll_18:51:102
165682431,38sleep216578-21ssh21:25:212
165682431,38sleep216578-21ssh21:25:212
132762430,3sleep30-21swapper/321:21:023
132762430,3sleep30-21swapper/321:21:023
2322899426,24cyclictest0-21swapper/317:06:013
2322899426,24cyclictest0-21swapper/317:06:013
2322799427,23cyclictest0-21swapper/219:35:552
2322799422,9cyclictest0-21swapper/218:36:002
23227994219,14cyclictest31-21ksoftirqd/221:01:142
23227994214,16cyclictest0-21swapper/220:11:132
2322699429,15cyclictest7750irq/56-dwc_otg20:36:011
2322699428,20cyclictest22909-21rm19:35:171
2322699428,13cyclictest4593-21grep17:06:161
2322699428,13cyclictest4593-21grep17:06:161
2322699427,18cyclictest7850irq/56-dwc_otg_18:38:401
23226994215,16cyclictest24527-21ssh20:55:401
23226994214,12cyclictest25669-21netstat20:56:101
23226994211,14cyclictest17494-21sh20:06:301
23226994210,7cyclictest25249-21ssh18:57:571
23225994225,7cyclictest0-21swapper/017:41:550
2322799417,25cyclictest0-21swapper/220:36:012
23227994120,11cyclictest301ktimersoftd/218:41:072
23227994119,5cyclictest7531-21latency_hist19:55:492
2322699417,22cyclictest0-21swapper/117:15:531
2322699416,14cyclictest25055-21runrttasks18:05:461
23226994114,12cyclictest13545-21ssh19:23:121
218692417,20sleep321963-21/usr/sbin/munin16:26:093
2322899401,27cyclictest23581-21ssh19:35:543
23228994010,14cyclictest7850irq/56-dwc_otg_19:51:153
2322799407,24cyclictest0-21swapper/220:46:022
23227994020,8cyclictest7750irq/56-dwc_otg18:06:062
23227994016,13cyclictest7850irq/56-dwc_otg_16:45:482
2322699401,24cyclictest0-21swapper/117:21:011
2322599401,22cyclictest0-21swapper/018:04:530
147582400,1sleep20-21swapper/220:04:032
23228993914,5cyclictest21447-21grep20:11:173
2322799399,14cyclictest7750irq/56-dwc_otg19:41:072
2322799397,25cyclictest0-21swapper/220:51:012
2322799396,24cyclictest0-21swapper/218:46:022
2322799396,23cyclictest0-21swapper/221:31:022
2322799393,12cyclictest0-21swapper/219:01:002
2322899381,25cyclictest10997-21cut17:26:053
2322899381,21cyclictest21935-21cut17:56:043
2322799386,8cyclictest31-21ksoftirqd/221:35:582
2322799384,26cyclictest31-21ksoftirqd/216:31:072
2322699385,12cyclictest0-21swapper/120:17:591
2322899378,12cyclictest7850irq/56-dwc_otg_18:36:153
2322899376,21cyclictest19874-21ls18:51:143
2322799376,24cyclictest0-21swapper/218:21:012
2322799375,20cyclictest4239-21ssh20:31:012
2322799372,7cyclictest7850irq/56-dwc_otg_16:55:482
2322799372,6cyclictest31-21ksoftirqd/219:26:072
2322699373,6cyclictest7850irq/56-dwc_otg_17:40:591
2322699371,22cyclictest0-21swapper/117:55:531
23226993711,15cyclictest32153-21taskset18:24:201
289472360,1sleep30-21swapper/321:00:533
2322899366,15cyclictest7850irq/56-dwc_otg_17:16:163
2322899365,15cyclictest7850irq/56-dwc_otg_21:36:023
2322899362,7cyclictest26117-21grep20:56:163
2322899362,20cyclictest0-21swapper/317:45:483
2322799367,17cyclictest7850irq/56-dwc_otg_21:09:312
23227993615,7cyclictest301ktimersoftd/217:06:092
23227993615,7cyclictest301ktimersoftd/217:06:092
23227993615,4cyclictest301ktimersoftd/221:26:172
23227993613,13cyclictest29-21rcuc/219:14:492
23227993613,13cyclictest29-21rcuc/219:14:492
2322799361,19cyclictest21554-21ls17:55:492
2322699367,13cyclictest1663-21/usr/sbin/munin18:30:481
96852350,2sleep29688-21aten_rbpower_vo21:56:032
2322899356,7cyclictest7850irq/56-dwc_otg_21:10:483
2322899356,14cyclictest7850irq/56-dwc_otg_17:51:013
2322899355,13cyclictest7850irq/56-dwc_otg_19:25:483
2322899353,16cyclictest7850irq/56-dwc_otg_20:31:013
2322899353,12cyclictest7850irq/56-dwc_otg_19:06:023
23228993518,5cyclictest14762-21/usr/sbin/munin18:46:023
2322899351,25cyclictest3203-21cut19:11:053
2322899351,25cyclictest3203-21cut19:11:053
2322799358,17cyclictest16964-21ssh20:06:132
2322799356,14cyclictest7850irq/56-dwc_otg_16:46:162
2322799355,21cyclictest7057-21ssh19:16:002
23227993517,5cyclictest18006-21basename17:46:012
23227993514,4cyclictest29-21rcuc/220:58:012
23227993513,12cyclictest0-21swapper/218:10:572
2322699351,6cyclictest7850irq/56-dwc_otg_17:45:591
2322699351,25cyclictest16017-21ls17:40:481
69882340,2sleep26985-21latency_hist17:15:492
2322899349,4cyclictest0-21swapper/321:20:493
2322899349,13cyclictest0-21swapper/320:21:013
23228993416,5cyclictest24294-21/usr/sbin/munin20:16:103
23227993417,9cyclictest301ktimersoftd/217:01:072
23227993414,9cyclictest32263-21sort19:46:052
23227993414,5cyclictest27088-21sh20:18:592
2322699347,23cyclictest0-21swapper/117:51:001
118242340,2sleep30-21swapper/321:58:023
2322899337,10cyclictest7950irq/56-dwc_otg_21:34:203
2322899336,10cyclictest0-21swapper/319:05:493
2322899335,7cyclictest7850irq/56-dwc_otg_17:01:033
2322899333,16cyclictest19516-21aten_rbpower_po19:31:023
2322899331,22cyclictest1579-21uniq21:46:053
23228993310,5cyclictest7950irq/56-dwc_otg_20:06:093
23227993325,6cyclictest23244-21latency_hist19:35:492
23227993317,4cyclictest31-21ksoftirqd/219:06:152
23227993312,12cyclictest31-21ksoftirqd/220:21:152
2322899328,5cyclictest7850irq/56-dwc_otg_21:41:043
2322899328,11cyclictest0-21swapper/321:11:513
2322899327,5cyclictest7750irq/56-dwc_otg19:16:043
2322899326,5cyclictest7950irq/56-dwc_otg_16:56:023
2322899324,5cyclictest7950irq/56-dwc_otg_16:41:173
2322899322,19cyclictest0-21swapper/317:40:373
2322899321,25cyclictest7054-21ls17:15:493
2322899321,24cyclictest15622-21cut19:26:053
2322899321,21cyclictest397-21cut20:26:063
23228993211,4cyclictest0-21swapper/320:51:423
2322799324,18cyclictest0-21swapper/217:31:012
2322799323,10cyclictest0-21swapper/219:20:592
23227993214,9cyclictest32306-21basename16:56:012
2322899317,4cyclictest7750irq/56-dwc_otg18:41:083
2322899311,8cyclictest13712-21sed17:31:163
23228993114,10cyclictest24099-21grep18:56:153
2322899311,19cyclictest18306-21grep17:46:053
2322799314,6cyclictest7850irq/56-dwc_otg_18:27:192
2322899306,12cyclictest7850irq/56-dwc_otg_18:01:083
23227993015,5cyclictest29-21rcuc/217:36:592
2322899298,4cyclictest0-21swapper/321:28:473
2322899295,14cyclictest0-21swapper/316:46:153
2322899293,4cyclictest0-21swapper/319:56:033
2322899292,6cyclictest23261-21aten_rbpower_cu16:31:013
2322899292,19cyclictest0-21swapper/318:06:473
2322899291,7cyclictest29203-21fschecks_time18:16:043
2322799296,5cyclictest31-21ksoftirqd/217:26:052
2322799296,14cyclictest0-21swapper/217:55:532
2322899284,5cyclictest0-21swapper/318:11:063
2322899283,13cyclictest7850irq/56-dwc_otg_18:25:083
2322899272,17cyclictest0-21swapper/317:21:563
226092273,15sleep20-21swapper/216:26:242
2322799266,12cyclictest908-21runrttasks18:01:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional