You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 08:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Jun 21, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
305682960,6sleep31873699cyclictest21:49:113
74092870,3sleep37412-21unixbench_singl20:05:333
88532820,9sleep0101ktimersoftd/021:23:070
53852800,11sleep01873399cyclictest20:00:310
153642760,2sleep30-21swapper/322:49:013
107612760,2sleep0101ktimersoftd/022:04:330
1873399756,57cyclictest0-21swapper/020:25:190
18733997417,40cyclictest26220-21fschecks_count19:30:220
1873399723,31cyclictest0-21swapper/021:40:170
1873399723,31cyclictest0-21swapper/021:40:170
1873399722,53cyclictest25345-21grep19:25:330
127812720,2sleep00-21swapper/023:25:140
127812720,2sleep00-21swapper/023:25:140
1873399718,40cyclictest125-21systemd-journal19:10:280
1873399705,42cyclictest28969-21cut23:05:220
1873399696,53cyclictest0-21swapper/020:30:160
1873599686,44cyclictest8050irq/56-dwc_otg_20:25:332
18733996817,30cyclictest0-21swapper/020:40:170
97452670,1sleep20-21swapper/220:15:062
1873599674,48cyclictest8050irq/56-dwc_otg_23:19:422
1873399674,51cyclictest32101-21ssh21:50:280
1873399673,48cyclictest9-21ksoftirqd/021:15:200
1873399671,49cyclictest1444-21cron21:15:060
1873399671,49cyclictest1444-21cron21:15:060
1873599661,4cyclictest8050irq/56-dwc_otg_19:55:172
1873399661,53cyclictest29616-21apt21:10:060
18733996613,44cyclictest0-21swapper/023:30:180
136402660,4sleep113632-21ntp_states23:25:311
136402660,4sleep113632-21ntp_states23:25:311
1873599655,44cyclictest8050irq/56-dwc_otg_21:40:282
1873599655,44cyclictest8050irq/56-dwc_otg_21:40:282
1873599651,50cyclictest11817-21/usr/sbin/munin20:20:202
1873599651,4cyclictest22655-21apt-get20:50:182
18733996537,8cyclictest12650-21unixbench_multi22:05:360
18733996511,35cyclictest125-21systemd-journal22:15:060
18733996511,35cyclictest125-21systemd-journal22:15:060
1873599644,4cyclictest19118-21aten_rbpower_po20:40:202
1873399645,51cyclictest0-21swapper/000:00:060
1873399644,48cyclictest11328-21ssh00:01:470
1873599635,43cyclictest8050irq/56-dwc_otg_21:50:332
1873399632,45cyclictest101ktimersoftd/022:45:230
180562630,6sleep121-21rcuc/119:05:351
180562630,6sleep121-21rcuc/119:05:351
1873599627,36cyclictest8050irq/56-dwc_otg_23:25:362
1873599627,36cyclictest8050irq/56-dwc_otg_23:25:362
1873599625,46cyclictest7850irq/56-dwc_otg22:40:062
1873599625,46cyclictest7850irq/56-dwc_otg22:40:062
1873599623,4cyclictest0-21swapper/221:45:342
1873599621,48cyclictest31525-21ssh23:08:212
1873599621,47cyclictest7850irq/56-dwc_otg00:12:112
1873599621,47cyclictest7850irq/56-dwc_otg00:12:112
1873399624,38cyclictest22739-21wget20:50:190
1873399622,49cyclictest0-21swapper/019:35:190
1873599611,49cyclictest4218-21ls21:55:332
1873599611,49cyclictest27460-21ssh00:20:482
1873399611,50cyclictest0-21swapper/022:35:330
1873399611,50cyclictest0-21swapper/022:35:330
1873399611,48cyclictest23548-21ssh00:16:150
1873599604,48cyclictest8050irq/56-dwc_otg_22:25:242
1873599602,5cyclictest7632-21ssh22:00:242
1873399606,34cyclictest27939-21/usr/sbin/munin22:25:160
1873399605,35cyclictest2507-21sort19:55:050
18733996015,29cyclictest18346-21/usr/sbin/munin00:10:270
18733996015,29cyclictest18346-21/usr/sbin/munin00:10:270
89992590,2sleep10-21swapper/100:00:051
1873599595,38cyclictest8050irq/56-dwc_otg_23:00:262
1873599592,41cyclictest0-21swapper/221:04:342
1873599591,8cyclictest9181-21ssh00:00:072
1873599591,5cyclictest25346-21switchtime20:55:312
1873599591,43cyclictest7850irq/56-dwc_otg00:03:112
1873399594,8cyclictest26025-21apt-get00:20:180
18733995934,13cyclictest0-21swapper/021:30:290
1873399591,43cyclictest2488-21ssh00:30:280
1873399591,32cyclictest0-21swapper/000:05:170
299542580,3sleep129957-21grep23:05:341
20592580,2sleep20-21swapper/222:32:342
18736995815,21cyclictest18281-21ssh22:13:103
18736995815,21cyclictest18281-21ssh22:13:103
1873599588,39cyclictest4633-21ssh21:18:102
1873599586,35cyclictest22641-21ssh22:57:122
1873599584,37cyclictest7850irq/56-dwc_otg23:50:192
1873599581,47cyclictest10553-21ssh22:42:572
1873599581,42cyclictest0-21swapper/221:11:532
1873599581,42cyclictest0-21swapper/221:11:532
1873399581,50cyclictest19495-21ssh22:53:590
1873599576,45cyclictest7950irq/56-dwc_otg_23:40:052
1873599573,40cyclictest0-21swapper/221:38:092
1873599572,39cyclictest11641-21seq22:05:222
1873599571,46cyclictest7850irq/56-dwc_otg19:25:322
18735995710,39cyclictest125-21systemd-journal21:10:062
1873399571,41cyclictest0-21swapper/019:15:180
174142570,2sleep20-21swapper/223:30:232
1873599561,5cyclictest17395-21ssh22:11:342
1873599561,5cyclictest17395-21ssh22:11:342
1873599561,5cyclictest15250-21ssh00:06:242
18735995615,38cyclictest16283-21/usr/sbin/munin22:50:262
1873599561,46cyclictest0-21swapper/200:26:342
1873399561,36cyclictest24005-21latency_hist19:25:060
293292550,2sleep10-21swapper/121:46:551
1873599553,43cyclictest0-21swapper/222:45:272
1873599551,4cyclictest0-21swapper/223:20:072
1873599551,44cyclictest0-21swapper/200:34:492
1873599551,43cyclictest0-21swapper/219:50:062
1873599551,43cyclictest0-21swapper/219:50:062
18733995537,4cyclictest101ktimersoftd/023:11:240
1873399552,43cyclictest0-21swapper/022:55:350
1873399552,29cyclictest0-21swapper/020:35:160
233632540,4sleep323360-21ssh21:40:213
233632540,4sleep323360-21ssh21:40:213
18736995415,16cyclictest13741-21ssh23:25:323
18736995415,16cyclictest13741-21ssh23:25:323
18736995413,12cyclictest24271-21sh23:00:063
1873599541,4cyclictest0-21swapper/221:30:232
1873599541,45cyclictest27087-21sed19:30:342
1873599541,42cyclictest8050irq/56-dwc_otg_00:35:122
1873399548,28cyclictest0-21swapper/000:25:190
1873399544,40cyclictest0-21swapper/022:20:260
1873399543,36cyclictest3016-21tune2fs19:55:230
18733995424,7cyclictest101ktimersoftd/022:33:510
1873399541,39cyclictest615-21sh19:50:060
1873399541,39cyclictest615-21sh19:50:060
1873399541,32cyclictest12021-21fschecks_time20:20:230
18733995411,27cyclictest101ktimersoftd/023:54:450
18736995315,17cyclictest19946-21ssh23:33:353
18736995313,14cyclictest7950irq/56-dwc_otg_00:00:163
1873599534,7cyclictest0-21swapper/223:45:162
1873599532,36cyclictest7950irq/56-dwc_otg_21:20:592
1873599531,5cyclictest1364-21ls19:50:262
18735995311,39cyclictest7263-21timerwakeupswit20:05:322
18734995313,20cyclictest3781-21ssh22:35:171
18734995313,20cyclictest3781-21ssh22:35:171
18733995334,8cyclictest0-21swapper/023:02:320
18733995329,14cyclictest101ktimersoftd/021:29:340
18733995327,12cyclictest101ktimersoftd/023:19:370
1873399532,34cyclictest8298-21fschecks_time22:40:230
1873399531,41cyclictest0-21swapper/023:45:050
1873399531,28cyclictest0-21swapper/021:55:240
1873699527,28cyclictest31445-21ssh21:50:203
18736995215,11cyclictest3192-21ssh21:15:423
18736995213,11cyclictest26276-21diskstats00:20:243
1873599527,32cyclictest7850irq/56-dwc_otg21:25:312
1873599522,5cyclictest8050irq/56-dwc_otg_20:20:062
1873599522,3cyclictest8050irq/56-dwc_otg_22:15:302
1873599521,47cyclictest24115-21fschecks_count22:20:232
1873599521,44cyclictest0-21swapper/223:11:112
1873599521,41cyclictest0-21swapper/200:17:562
1873599521,3cyclictest0-21swapper/219:36:032
1873499526,30cyclictest26042-21ssh00:20:211
18733995210,27cyclictest0-21swapper/000:40:060
1873599518,39cyclictest15468-21/usr/sbin/munin20:30:182
1873599512,37cyclictest0-21swapper/219:11:592
1873599511,4cyclictest25247-21apt-get23:40:182
18735995111,37cyclictest18696-21latency_hist20:40:062
1873399512,33cyclictest0-21swapper/021:45:170
1873399511,46cyclictest8658-21ls20:10:260
18733995112,27cyclictest29720-21cut19:40:170
1873599502,45cyclictest22436-21latency_hist20:50:052
18733995030,10cyclictest24846-21ssh23:39:590
18733995011,22cyclictest26510-21fschecks_count21:00:220
18733995010,30cyclictest101ktimersoftd/022:16:480
1873599496,41cyclictest24006-21latency_hist19:25:062
1873599491,45cyclictest30855-21grep19:40:322
1873399493,21cyclictest6516-21fschecks_count20:05:210
18733994927,10cyclictest7950irq/56-dwc_otg_20:15:260
18733994924,9cyclictest31007-21rm23:46:070
18736994816,12cyclictest25586-21grep23:40:233
18736994810,22cyclictest7850irq/56-dwc_otg22:54:013
1873599481,44cyclictest20822-21cut19:15:222
182752483,13sleep00-21swapper/019:07:370
182752483,13sleep00-21swapper/019:07:370
1873699479,23cyclictest29617-21diskmemload23:02:573
1873699479,23cyclictest24937-21/usr/sbin/munin19:25:303
18736994726,11cyclictest20832-21taskset20:45:143
18736994721,4cyclictest8050irq/56-dwc_otg_23:05:273
18736994714,19cyclictest9955-21ssh22:03:043
18736994712,24cyclictest11687-21munin-node20:20:163
1873599472,42cyclictest6044-21munin-run20:05:052
18733994726,8cyclictest101ktimersoftd/021:35:330
1873699468,19cyclictest0-21swapper/321:12:013
1873699468,19cyclictest0-21swapper/321:12:013
1873699467,25cyclictest0-21swapper/320:30:163
1873699467,25cyclictest0-21swapper/320:25:193
18736994615,7cyclictest22864-21df_inode20:50:213
18736994612,19cyclictest7950irq/56-dwc_otg_19:20:053
18736994610,15cyclictest31779-21ssh00:26:093
1873499466,26cyclictest0-21swapper/100:10:201
1873499466,26cyclictest0-21swapper/100:10:201
18733994634,5cyclictest7950irq/56-dwc_otg_23:20:330
18733994610,24cyclictest0-21swapper/020:45:200
1873699458,19cyclictest28208-21diskstats22:25:223
1873699457,24cyclictest0-21swapper/319:50:193
18736994516,21cyclictest14483-21ssh22:08:483
18736994515,17cyclictest18556-21idleruntime-cro21:35:063
18736994513,15cyclictest23640-21latency_hist22:20:063
1873499457,24cyclictest0-21swapper/119:50:191
1873499457,24cyclictest0-21swapper/119:10:191
1873699449,22cyclictest11342-21ssh23:23:003
1873699448,20cyclictest21575-21ssh21:37:583
1873699447,24cyclictest0-21swapper/320:40:113
1873699445,23cyclictest8050irq/56-dwc_otg_21:20:573
18736994422,9cyclictest8050irq/56-dwc_otg_00:30:313
18736994414,14cyclictest12386-21ssh21:26:493
18736994413,16cyclictest2829-21ssh22:33:583
18736994413,11cyclictest30832-21ssh23:45:483
18736994411,19cyclictest25975-21/usr/sbin/munin19:30:353
1873499448,24cyclictest0-21swapper/119:45:181
1873499448,24cyclictest0-21swapper/119:45:181
1873499446,25cyclictest18898-21ssh21:35:191
1873499446,24cyclictest0-21swapper/120:45:201
1873499441,3cyclictest0-21swapper/119:40:061
1873699439,16cyclictest0-21swapper/300:20:013
1873699438,18cyclictest26382-21ssh22:22:503
18736994311,22cyclictest18639-21runrttasks20:39:353
18734994325,12cyclictest23-21ksoftirqd/122:05:321
1873699426,16cyclictest8050irq/56-dwc_otg_19:55:323
18736994213,19cyclictest4565-21ssh23:54:293
18736994210,18cyclictest30960-21sshd19:41:283
18736994210,17cyclictest7552-21chrt22:40:033
18736994210,17cyclictest7552-21chrt22:40:033
1873499427,27cyclictest0-21swapper/121:40:191
1873499427,27cyclictest0-21swapper/121:40:191
1873499426,25cyclictest0-21swapper/122:10:201
1873499426,25cyclictest0-21swapper/122:10:201
1873499426,24cyclictest0-21swapper/120:40:111
1873499421,13cyclictest0-21swapper/121:30:181
1873699418,17cyclictest2036-21ssh23:11:093
1873699415,24cyclictest4564-21ssh23:15:203
18736994114,13cyclictest8050irq/56-dwc_otg_23:37:073
18736994113,15cyclictest21460-21ssh00:14:593
18736994113,15cyclictest21460-21ssh00:14:593
18736994111,12cyclictest5324-21ssh21:57:113
18736994110,12cyclictest10325-21sh22:42:333
1873499417,24cyclictest0-21swapper/122:15:181
1873499415,23cyclictest0-21swapper/121:15:171
18733994123,5cyclictest7950irq/56-dwc_otg_20:58:010
1873699407,24cyclictest0-21swapper/320:55:193
1873699401,31cyclictest8050irq/56-dwc_otg_00:05:183
1873499407,27cyclictest0-21swapper/120:35:181
1873499406,24cyclictest0-21swapper/123:40:191
1873499406,24cyclictest0-21swapper/119:20:181
18734994021,8cyclictest23-21ksoftirqd/119:55:221
18734994018,15cyclictest221ktimersoftd/123:00:061
18736993912,11cyclictest29038-21seq19:36:193
1873499396,24cyclictest0-21swapper/120:20:191
1873499395,25cyclictest27973-21ssh22:25:191
1873499392,26cyclictest0-21swapper/120:50:161
18734993916,11cyclictest21-21rcuc/123:45:381
1873699385,19cyclictest7950irq/56-dwc_otg_20:14:283
1873699385,16cyclictest295772chrt21:09:493
18736993811,14cyclictest8582-21ssh00:37:563
18736993811,11cyclictest7510-21ssh23:57:213
1873499387,28cyclictest0-21swapper/123:30:181
1873499386,23cyclictest0-21swapper/100:35:181
18734993817,12cyclictest221ktimersoftd/119:25:301
1873499376,5cyclictest4561-21aten_rbpower_po20:00:181
1873499375,6cyclictest7950irq/56-dwc_otg_00:00:341
1873499374,4cyclictest8050irq/56-dwc_otg_00:30:371
1873499371,8cyclictest11701-21ssh22:45:051
18734993718,5cyclictest21-21rcuc/123:14:101
18734993715,5cyclictest21-21rcuc/121:09:381
18734993712,15cyclictest221ktimersoftd/123:51:341
1873699369,12cyclictest20036-21taskset19:11:393
1873699369,11cyclictest750-21snmpd20:15:403
18736993610,11cyclictest5891-21taskset20:03:233
1873499367,23cyclictest0-21swapper/120:15:191
1873499366,7cyclictest23-21ksoftirqd/120:25:231
1873499365,20cyclictest7293-21ssh22:00:191
1873499363,12cyclictest8050irq/56-dwc_otg_21:55:231
18734993620,9cyclictest221ktimersoftd/123:40:051
18734993612,4cyclictest0-21swapper/122:24:501
18734993611,12cyclictest13753-21grep00:05:231
18734993610,4cyclictest0-21swapper/122:45:101
1873699358,16cyclictest328-21runrttasks19:46:543
1873699358,16cyclictest328-21runrttasks19:46:543
1873499353,27cyclictest8050irq/56-dwc_otg_23:20:161
1873499351,6cyclictest8050irq/56-dwc_otg_00:15:231
18734993511,10cyclictest9650irq/80-mmc020:15:051
1873499349,12cyclictest20557-21/usr/sbin/munin19:15:211
1873499349,12cyclictest20271-21latency_hist22:55:061
1873499346,18cyclictest0-21swapper/120:56:441
1873499344,6cyclictest727-21runrttasks21:50:191
1873499344,14cyclictest21-21rcuc/119:30:271
18734993413,5cyclictest23-21ksoftirqd/121:26:411
1873699337,16cyclictest8050irq/56-dwc_otg_19:22:283
1873499337,4cyclictest8050irq/56-dwc_otg_23:15:361
1873499332,16cyclictest8050irq/56-dwc_otg_22:30:301
18734993320,4cyclictest727-21runrttasks00:28:181
18734993312,10cyclictest7372-21grep20:05:331
1873499328,4cyclictest6221-21cut21:20:201
1873499324,6cyclictest21-21rcuc/121:01:191
1873499322,5cyclictest7850irq/56-dwc_otg19:40:171
18734993213,10cyclictest26622-21ssh23:01:561
18734993211,4cyclictest0-21swapper/121:10:181
18734993211,4cyclictest0-21swapper/121:10:181
172692326,13sleep20-21swapper/219:05:252
172692326,13sleep20-21swapper/219:05:252
18736993117,11cyclictest27163-21/usr/sbin/munin21:00:313
1873499319,20cyclictest0-21swapper/120:30:161
182602264,15sleep30-21swapper/319:07:283
182602264,15sleep30-21swapper/319:07:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional