You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 03:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Aug 31, 2026 00:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1612225660,1sleep10-21swapper/119:05:421
1612225660,1sleep10-21swapper/119:05:421
1682022960,3sleep30-21swapper/319:10:133
1682022960,3sleep30-21swapper/319:10:133
169469912231,16cyclictest0-21swapper/022:20:410
169469912231,16cyclictest0-21swapper/022:20:410
169469912018,36cyclictest30946-21unixbench_singl22:30:470
1694899953,76cyclictest0-21swapper/222:20:422
1694899953,76cyclictest0-21swapper/222:20:422
16946999455,11cyclictest101ktimersoftd/023:05:370
64252920,2sleep10-21swapper/122:01:071
1694999893,26cyclictest0-21swapper/322:55:173
250472870,2sleep30-21swapper/321:00:303
16948998716,51cyclictest0-21swapper/222:55:202
1694699873,65cyclictest0-21swapper/022:55:190
175842820,3sleep20-21swapper/220:40:282
1694699815,12cyclictest60-21kswapd023:20:250
1694899803,67cyclictest0-21swapper/223:05:362
199012780,1sleep10-21swapper/121:39:081
1694899783,12cyclictest30937-21ssh22:30:472
1694799784,12cyclictest0-21swapper/122:30:461
84722740,5sleep11694799cyclictest22:43:541
253022740,3sleep3381ktimersoftd/323:43:233
308742710,11sleep00-21swapper/019:45:400
1694899697,5cyclictest2752-21fschecks_count23:55:312
1694899697,5cyclictest2752-21fschecks_count23:55:312
27702670,4sleep02763-21apt-config00:35:140
1694899671,6cyclictest6676-21ssh22:40:412
16946996722,23cyclictest22694-21wget19:25:270
1694899668,4cyclictest0-21swapper/222:50:052
1694899668,4cyclictest0-21swapper/222:50:052
1694899655,9cyclictest7950irq/56-dwc_otg_21:15:292
16948996410,45cyclictest18363-21ssh23:35:232
219852630,2sleep20-21swapper/200:18:532
1694899632,5cyclictest7750irq/56-dwc_otg19:50:272
1694899631,12cyclictest7950irq/56-dwc_otg_21:35:332
1694899626,42cyclictest7850irq/56-dwc_otg_00:35:132
1694899623,4cyclictest7750irq/56-dwc_otg22:00:302
1694899622,4cyclictest7750irq/56-dwc_otg20:25:302
1694899621,5cyclictest7950irq/56-dwc_otg_20:20:312
1694899604,9cyclictest1180-21runrttasks22:11:542
1694899602,7cyclictest4006-21grep00:35:382
1694899601,5cyclictest13744-21/usr/sbin/munin20:30:322
1694699601,27cyclictest25189-21ls21:00:320
216462590,6sleep121076-21/usr/sbin/munin20:50:341
1694999598,17cyclictest7750irq/56-dwc_otg22:20:423
1694999598,17cyclictest7750irq/56-dwc_otg22:20:423
1694899593,12cyclictest2987-21ssh21:57:222
1694899592,5cyclictest0-21swapper/219:42:132
16946995929,13cyclictest7850irq/56-dwc_otg_21:30:480
16946995923,6cyclictest12294-21sh21:30:140
16946995910,32cyclictest0-21swapper/021:05:310
1694899585,8cyclictest7950irq/56-dwc_otg_20:45:292
1694899581,9cyclictest7750irq/56-dwc_otg22:06:322
1694899581,42cyclictest7950irq/56-dwc_otg_23:15:322
1694699582,50cyclictest0-21swapper/020:15:300
16946995820,26cyclictest14408-21wget23:30:280
1694699581,23cyclictest0-21swapper/019:35:320
1694899576,4cyclictest7750irq/56-dwc_otg21:40:482
1694899574,4cyclictest7950irq/56-dwc_otg_19:20:372
1694899572,5cyclictest0-21swapper/219:15:132
1694899571,5cyclictest31161-21ssh21:52:052
1694899571,5cyclictest31161-21ssh21:52:052
1694899571,12cyclictest9920-21ssh00:04:282
1694799573,21cyclictest11-21rcu_preempt22:55:191
1694799573,13cyclictest26080-21/usr/sbin/munin23:05:361
16946995728,6cyclictest11388-21grep23:25:420
16946995728,6cyclictest11388-21grep23:25:420
68072560,2sleep10-21swapper/121:22:461
1694899567,7cyclictest7950irq/56-dwc_otg_21:26:422
16948995612,41cyclictest25963-21cron19:35:132
16948995612,41cyclictest25963-21cron19:35:132
1694799563,13cyclictest0-21swapper/122:20:441
1694799563,13cyclictest0-21swapper/122:20:441
1694899558,4cyclictest0-21swapper/221:47:242
1694899552,3cyclictest0-21swapper/222:29:322
1694899551,10cyclictest30797-21taskset00:29:572
1694699552,38cyclictest0-21swapper/022:45:240
1694699552,38cyclictest0-21swapper/022:45:240
16946995520,15cyclictest6550-21/usr/sbin/munin00:00:300
1694699551,27cyclictest5096-21awk22:40:130
1694999545,33cyclictest0-21swapper/322:30:483
16949995419,18cyclictest0-21swapper/322:05:353
16949995412,23cyclictest26869-21/usr/sbin/munin23:05:373
1694899547,3cyclictest0-21swapper/223:00:442
1694899545,8cyclictest7950irq/56-dwc_otg_00:09:522
1694899542,10cyclictest16064-21grep00:10:432
1694899541,7cyclictest17329-21ssh22:55:032
16948995411,8cyclictest0-21swapper/221:20:262
1694699547,27cyclictest1695-21latency_hist23:15:140
16946995429,6cyclictest10255-21ssh22:05:430
1694699542,8cyclictest29193-21unixbench_singl19:40:420
16946995414,27cyclictest101ktimersoftd/000:15:290
1694899534,7cyclictest7950irq/56-dwc_otg_19:36:492
1694899533,11cyclictest7950irq/56-dwc_otg_21:35:132
1694899532,3cyclictest7950irq/56-dwc_otg_22:17:392
1694899532,3cyclictest0-21swapper/220:50:382
1694799535,7cyclictest7950irq/56-dwc_otg_00:35:251
16946995332,12cyclictest4663-21fschecks_count21:20:320
16946995326,10cyclictest7950irq/56-dwc_otg_23:45:130
16946995323,3cyclictest0-21swapper/020:00:260
1694699532,27cyclictest15323-21sh00:10:340
16946995322,5cyclictest18716-21fschecks_count23:35:320
1694699531,7cyclictest18943-21munin-run20:45:130
1694699531,3cyclictest0-21swapper/023:50:140
1694699531,32cyclictest13621-21ls20:30:130
1694699531,28cyclictest6240-21latency_hist20:10:140
1694899524,7cyclictest7950irq/56-dwc_otg_19:56:152
1694899522,9cyclictest7950irq/56-dwc_otg_19:17:562
1694899522,41cyclictest0-21swapper/223:47:152
1694899521,10cyclictest0-21swapper/223:41:312
16946995227,6cyclictest7850irq/56-dwc_otg_21:45:270
1694699522,6cyclictest3203450ssh21:14:450
16946995222,10cyclictest17434-21apt-get22:55:130
1694699521,7cyclictest1100-21ssh21:55:310
1694699521,34cyclictest31517-21systemd-journal00:00:130
1694699521,34cyclictest31517-21systemd-journal00:00:130
1694699521,26cyclictest72650ssh21:15:340
1694699521,26cyclictest27526-21fschecks_time00:25:300
1694699521,25cyclictest32049-21grep23:50:420
1694899513,3cyclictest0-21swapper/200:20:262
1694899512,9cyclictest0-21swapper/223:27:012
1694899512,9cyclictest0-21swapper/223:27:012
1694899512,13cyclictest0-21swapper/219:25:252
1694899511,6cyclictest7950irq/56-dwc_otg_21:10:132
1694899511,3cyclictest0-21swapper/221:10:562
16947995114,13cyclictest27164-21ssh00:25:241
1694699516,25cyclictest1097-21aten_rbpower_vo19:55:280
1694699516,24cyclictest0-21swapper/020:30:270
16946995124,3cyclictest0-21swapper/023:00:260
1694699511,9cyclictest21455-21ps19:20:400
16946995118,7cyclictest0-21swapper/021:40:250
1694699511,33cyclictest26951-21ls00:25:140
1694699511,29cyclictest17387-21aten_rbpower_vo22:15:280
1694899501,4cyclictest32122-21ssh23:50:462
1694899501,3cyclictest0-21swapper/223:31:112
1694899501,13cyclictest0-21swapper/220:56:272
1694899501,12cyclictest0-21swapper/223:12:142
1694699503,35cyclictest0-21swapper/020:55:410
1694699503,26cyclictest0-21swapper/019:10:330
16946995026,5cyclictest5724-21grep22:00:370
16946995023,12cyclictest20746-21apt-get20:50:130
1694699501,25cyclictest0-21swapper/022:25:490
1694899495,7cyclictest7950irq/56-dwc_otg_22:37:472
1694899492,7cyclictest7950irq/56-dwc_otg_20:36:542
1694899491,8cyclictest2894-21wget20:00:282
1694899491,3cyclictest7850irq/56-dwc_otg_23:23:512
16946994923,4cyclictest29419-21fschecks_time21:50:310
16946994923,4cyclictest29419-21fschecks_time21:50:310
16946994913,17cyclictest0-21swapper/019:30:250
16946994913,17cyclictest0-21swapper/019:30:250
1694699491,26cyclictest10814-21grep20:20:360
1694899481,3cyclictest7950irq/56-dwc_otg_20:15:022
1694799482,9cyclictest0-21swapper/123:45:271
16947994813,7cyclictest7850irq/56-dwc_otg_21:10:271
16946994829,2cyclictest0-21swapper/000:05:250
16946994815,5cyclictest0-21swapper/019:15:250
1694699481,30cyclictest32160-21latency19:50:330
1694699479,21cyclictest0-21swapper/021:35:350
165822473,14sleep00-21swapper/019:08:580
165822473,14sleep00-21swapper/019:08:580
1694899467,25cyclictest0-21swapper/221:00:252
1694899467,24cyclictest0-21swapper/220:05:262
1694899461,10cyclictest9420-21grep20:15:412
1694799468,5cyclictest7950irq/56-dwc_otg_23:15:271
1694799467,25cyclictest0-21swapper/119:10:261
1694699467,22cyclictest9-21ksoftirqd/020:50:300
1694699464,12cyclictest0-21swapper/022:15:110
16946994622,4cyclictest5692-21rm22:40:290
16946994618,15cyclictest5690-21latency_hist23:20:140
16946994616,5cyclictest0-21swapper/000:35:230
16946994610,24cyclictest0-21swapper/020:10:260
1694999457,24cyclictest0-21swapper/320:10:263
16949994524,12cyclictest17433-21idleruntime-cro22:55:133
1694799452,8cyclictest7950irq/56-dwc_otg_19:20:271
1694899442,7cyclictest29818-21/usr/sbin/munin19:45:392
16947994411,10cyclictest7750irq/56-dwc_otg23:50:241
1694999437,25cyclictest0-21swapper/321:05:273
1694799435,5cyclictest7750irq/56-dwc_otg22:10:271
1694999422,17cyclictest381ktimersoftd/323:00:263
1694799411,6cyclictest7950irq/56-dwc_otg_20:45:271
185422400,1sleep30-21swapper/322:15:463
1694999407,20cyclictest0-21swapper/321:25:283
16949994019,5cyclictest9375-21cat22:45:143
1694999401,11cyclictest7950irq/56-dwc_otg_00:30:443
1694999401,10cyclictest19297-21cut00:15:323
1694699402,18cyclictest0-21swapper/020:35:270
321102390,2sleep10-21swapper/100:30:401
1694999391,30cyclictest24687-21ls21:45:133
1694799398,12cyclictest10677-21ssh00:05:251
1694799391,29cyclictest26156-21ls22:25:361
130812390,1sleep30-21swapper/300:07:433
1694999386,20cyclictest0-21swapper/300:10:273
1694799388,15cyclictest7950irq/56-dwc_otg_20:40:261
16947993821,4cyclictest22723-21idleruntime-cro00:20:131
16947993816,5cyclictest17692-21latency_hist22:55:141
1694999372,30cyclictest0-21swapper/323:55:253
1694999372,30cyclictest0-21swapper/323:55:253
1694999371,24cyclictest11776-21ls20:25:133
1694799377,4cyclictest7950irq/56-dwc_otg_22:35:271
1694799373,7cyclictest178750ssh21:55:401
16947993716,5cyclictest28063-21diskmemload21:26:531
16947993716,18cyclictest0-21swapper/123:10:251
16949993618,4cyclictest1180-21runrttasks23:39:313
16949993615,5cyclictest18939-21idleruntime-cro20:45:133
1694999361,20cyclictest29608-21irqstats21:50:343
1694999361,20cyclictest29608-21irqstats21:50:343
1694999361,16cyclictest0-21swapper/321:15:313
1694799368,6cyclictest7950irq/56-dwc_otg_21:50:271
1694799368,6cyclictest7950irq/56-dwc_otg_21:50:271
1694799368,11cyclictest7850irq/56-dwc_otg_23:30:261
16947993619,6cyclictest28063-21diskmemload00:00:391
1694999358,14cyclictest0-21swapper/320:55:273
16949993522,7cyclictest7950irq/56-dwc_otg_22:30:133
1694999352,22cyclictest13796-21ls21:30:473
1694999351,4cyclictest7750irq/56-dwc_otg19:30:253
1694999351,4cyclictest7750irq/56-dwc_otg19:30:253
1694999351,23cyclictest2301-21ls22:35:443
1694799359,6cyclictest0-21swapper/123:20:311
1694799357,23cyclictest0-21swapper/120:00:271
1694799356,4cyclictest7950irq/56-dwc_otg_23:26:481
1694799356,4cyclictest7950irq/56-dwc_otg_23:26:481
1694799356,18cyclictest0-21swapper/123:35:261
1694799356,15cyclictest7950irq/56-dwc_otg_21:00:131
16947993516,5cyclictest20510-21latency_hist19:20:131
16947993515,5cyclictest15825-21/usr/sbin/munin00:10:411
16947993514,9cyclictest14816-21sh21:32:211
16947993513,15cyclictest221ktimersoftd/121:16:141
1694999347,15cyclictest0-21swapper/319:25:203
1694999345,10cyclictest7750irq/56-dwc_otg23:52:473
1694999343,6cyclictest0-21swapper/320:15:413
1694999342,16cyclictest7850irq/56-dwc_otg_23:10:263
1694999341,5cyclictest7850irq/56-dwc_otg_20:00:263
16949993413,13cyclictest1180-21runrttasks23:31:493
1694999341,23cyclictest0-21swapper/319:50:263
1694799345,14cyclictest7950irq/56-dwc_otg_19:30:331
1694799345,14cyclictest7950irq/56-dwc_otg_19:30:331
16947993417,6cyclictest22942-21ssh21:42:081
1694799341,6cyclictest1624250snmpd21:01:501
16947993416,9cyclictest25663-21sh00:23:061
1694799341,5cyclictest0-21swapper/121:05:421
16947993413,5cyclictest10674-21sh22:06:211
16947993413,11cyclictest2898-21/usr/sbin/munin23:55:321
16947993413,11cyclictest2898-21/usr/sbin/munin23:55:321
1694799341,23cyclictest0-21swapper/120:05:261
1694799341,22cyclictest11703-21latency_hist20:25:131
64102330,3sleep337-21rcuc/323:20:323
1694999334,12cyclictest7950irq/56-dwc_otg_19:20:443
1694999331,7cyclictest7850irq/56-dwc_otg_19:40:133
1694999331,25cyclictest30000-21ls19:45:283
1694999331,22cyclictest7750irq/56-dwc_otg23:15:143
1694799334,4cyclictest7750irq/56-dwc_otg22:15:431
1694799334,13cyclictest7850irq/56-dwc_otg_23:00:291
1694799332,6cyclictest15796-21fschecks_count20:35:291
1694799332,13cyclictest25565-21ssh21:45:341
1694799331,8cyclictest16242-21snmpd20:15:281
1694799331,22cyclictest28287-21grep19:40:301
166212333,21sleep20-21swapper/219:09:282
166212333,21sleep20-21swapper/219:09:282
1694999326,11cyclictest7950irq/56-dwc_otg_23:25:433
1694999326,11cyclictest7950irq/56-dwc_otg_23:25:433
16949993220,4cyclictest8110-21sh00:01:083
16949993219,7cyclictest7850irq/56-dwc_otg_21:22:183
1694999321,25cyclictest17332-21ls19:10:313
1694999321,21cyclictest14126-21switchtime22:10:413
1694799324,7cyclictest7750irq/56-dwc_otg19:50:411
1694799322,14cyclictest7950irq/56-dwc_otg_20:25:411
1694799321,21cyclictest2005-21timerwakeupswit19:55:421
1694999317,20cyclictest0-21swapper/319:40:273
1694999312,17cyclictest0-21swapper/322:03:123
1694999311,28cyclictest27445-21cut00:25:303
1694999311,20cyclictest25608-21missed_timers21:45:353
16949993110,11cyclictest0-21swapper/323:45:253
1694799317,22cyclictest0-21swapper/119:45:261
1694799313,5cyclictest7750irq/56-dwc_otg19:25:391
1694999309,4cyclictest0-21swapper/321:55:363
1694999308,5cyclictest7850irq/56-dwc_otg_22:45:273
1694999308,5cyclictest7850irq/56-dwc_otg_22:45:273
1694999305,16cyclictest7850irq/56-dwc_otg_00:25:103
1694999305,13cyclictest7950irq/56-dwc_otg_20:50:133
1694999303,14cyclictest7950irq/56-dwc_otg_19:15:423
16949993014,5cyclictest19876-21ssh21:39:053
1694999301,26cyclictest32500-21ls21:15:143
1694799304,13cyclictest7950irq/56-dwc_otg_20:30:341
1694799302,15cyclictest0-21swapper/123:40:361
1694999294,4cyclictest7750irq/56-dwc_otg00:35:343
1694999291,7cyclictest6164-21modprobe20:10:133
1694999291,6cyclictest7950irq/56-dwc_otg_20:35:263
1694799296,11cyclictest7950irq/56-dwc_otg_22:45:331
1694799296,11cyclictest7950irq/56-dwc_otg_22:45:331
1694799293,4cyclictest26219-21/usr/sbin/munin19:35:241
1694799292,13cyclictest7950irq/56-dwc_otg_20:10:411
1694999281,16cyclictest0-21swapper/320:00:143
1694999277,18cyclictest0-21swapper/320:50:203
1694999271,7cyclictest12182-21fschecks_count20:25:303
1694999271,11cyclictest7750irq/56-dwc_otg20:30:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional