You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 11:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Tue Dec 30, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1315899250,24cyclictest0-21swapper/319:50:403
1315899241,2cyclictest11969-21cron21:20:013
1315199240,23cyclictest29975-21idleruntime-cro20:50:011
1315199240,22cyclictest313-21systemd-journal21:35:011
13158992322,0cyclictest0-21swapper/321:00:413
1315899231,21cyclictest622-21gdbus20:15:003
1315899230,2cyclictest4617-21apt00:25:003
1315899230,22cyclictest32357-21basename20:55:003
1315499230,22cyclictest22440-21apt-get23:55:422
1315199230,22cyclictest19574-21perf20:30:011
13148992320,2cyclictest9388-21cron00:35:010
13148992320,2cyclictest7028-21turbostat.cron00:30:000
1314899231,1cyclictest0-21swapper/022:00:000
13158992221,1cyclictest9727-21apt-get00:35:453
13158992221,1cyclictest8034-21apt-get23:25:413
13158992220,1cyclictest2377-21sendmail21:00:003
1315899220,2cyclictest31232-21turbostat22:00:003
1315899220,21cyclictest21015-21apt-get19:25:393
1315899220,21cyclictest19520-21apt-get21:35:403
1315899220,21cyclictest18710-21apt-get22:40:023
13154992220,1cyclictest2421-21sendmail-msp21:00:002
13154992220,1cyclictest0-21swapper/220:10:002
1315499220,21cyclictest7718-21turbostat.cron23:25:002
1315499220,21cyclictest2202-21sh00:20:002
1315499220,21cyclictest0-21swapper/221:30:392
13151992220,1cyclictest0-21swapper/122:00:001
13151992220,1cyclictest0-21swapper/100:10:011
13151992219,2cyclictest20873-21latency_hist22:45:001
1315199221,0cyclictest25810-21apt-get19:35:391
1315199220,21cyclictest9905-21apt-get21:15:401
1315199220,21cyclictest15083-21apt-get20:20:411
1315199220,21cyclictest0-21swapper/123:50:411
1315199220,21cyclictest0-21swapper/123:50:411
1315199220,21cyclictest0-21swapper/123:30:401
1315199220,21cyclictest0-21swapper/122:50:401
1315199220,21cyclictest0-21swapper/122:25:411
1315199220,21cyclictest0-21swapper/121:05:391
1315199220,1cyclictest25513-21latency_hist19:35:011
1315199220,1cyclictest15981-21apt-get22:35:411
1315199220,1cyclictest12326-21apt-get21:20:421
1315199220,0cyclictest0-21swapper/123:45:421
13148992222,0cyclictest0-21swapper/020:25:410
13148992220,1cyclictest7562-21basename20:05:000
13148992219,2cyclictest7737-21latency_hist23:25:000
1314899220,21cyclictest0-21swapper/023:55:010
1314899220,21cyclictest0-21swapper/023:55:010
1314899220,21cyclictest0-21swapper/022:55:010
1314899220,21cyclictest0-21swapper/022:40:020
1314899220,21cyclictest0-21swapper/022:25:410
1314899220,21cyclictest0-21swapper/021:50:430
1314899220,21cyclictest0-21swapper/021:10:430
1314899220,21cyclictest0-21swapper/020:35:020
1314899220,21cyclictest0-21swapper/019:40:400
1314899220,1cyclictest27875-21apt-get20:45:440
13158992120,1cyclictest24947-21apt-get00:00:433
13158992120,1cyclictest23580-21apt-get22:50:403
13158992119,1cyclictest30501-21grep23:05:013
13158992119,1cyclictest14954-21perf23:40:003
1315899210,20cyclictest6367-21apt-get22:15:413
1315899210,20cyclictest331rcuc/323:25:003
1315899210,20cyclictest24333-21apt-get21:45:413
1315899210,20cyclictest0-21swapper/323:40:433
1315899210,0cyclictest0-21swapper/322:45:403
1315899210,0cyclictest0-21swapper/319:40:403
13154992121,0cyclictest28228-21apt-get19:40:402
13154992120,1cyclictest5473-21apt-get20:00:412
13154992120,1cyclictest30270-21apt-get20:50:422
1315499210,21cyclictest8787-21apt-get22:20:412
1315499210,21cyclictest19875-21apt-get20:30:422
1315499210,20cyclictest555-21apt-get19:50:402
1315499210,0cyclictest21940-21apt-get21:40:422
1315499210,0cyclictest0-21swapper/223:10:392
1315499210,0cyclictest0-21swapper/219:10:022
13151992121,0cyclictest745-21apt-get23:10:381
13151992121,0cyclictest28398-21apt-get23:00:421
13151992120,1cyclictest0-21swapper/100:20:441
13151992119,1cyclictest31540-21apt-get22:00:391
1315199210,20cyclictest0-21swapper/120:35:401
1315199210,20cyclictest0-21swapper/119:10:021
1315199210,0cyclictest0-21swapper/122:30:411
1315199210,0cyclictest0-21swapper/121:25:401
13148992121,0cyclictest0-21swapper/020:30:420
13148992120,1cyclictest24038-21perf21:45:000
13148992120,1cyclictest0-21swapper/022:30:410
1314899210,20cyclictest17125-21apt-get21:30:390
1314899210,20cyclictest0-21swapper/019:10:410
1314899210,1cyclictest12824-21apt-get23:35:410
13158992020,0cyclictest0-21swapper/323:55:423
13158992020,0cyclictest0-21swapper/322:10:413
13158992020,0cyclictest0-21swapper/322:10:413
13158992020,0cyclictest0-21swapper/321:40:423
13158992020,0cyclictest0-21swapper/320:30:423
13158992020,0cyclictest0-21swapper/319:35:393
13158992020,0cyclictest0-21swapper/300:30:413
13158992019,1cyclictest16205-21apt-get19:15:403
13158992019,0cyclictest1516-21apt-get22:05:433
13158992018,1cyclictest7516-21apt-get21:10:433
1315899200,20cyclictest0-21swapper/323:05:413
1315899200,0cyclictest15981-21apt-get22:35:413
1315899200,0cyclictest0-21swapper/321:30:393
1315899200,0cyclictest0-21swapper/320:15:383
13154992020,0cyclictest26731-21apt-get21:50:432
13154992020,0cyclictest0-21swapper/222:55:412
13154992020,0cyclictest0-21swapper/222:40:412
13154992019,1cyclictest12649-21apt-get20:15:382
1315499200,20cyclictest0-21swapper/223:30:402
1315499200,20cyclictest0-21swapper/222:50:402
1315499200,20cyclictest0-21swapper/221:50:002
1315499200,0cyclictest0-21swapper/223:25:412
1315499200,0cyclictest0-21swapper/223:00:422
1315499200,0cyclictest0-21swapper/220:35:022
13151992020,0cyclictest0-21swapper/123:35:411
13151992020,0cyclictest0-21swapper/120:55:401
13151992020,0cyclictest0-21swapper/120:50:421
13151992020,0cyclictest0-21swapper/119:55:421
13151992020,0cyclictest0-21swapper/100:15:021
13151992019,0cyclictest0-21swapper/122:45:411
1315199200,0cyclictest10260-21apt-get20:10:411
1315199200,0cyclictest0-21swapper/122:10:411
1315199200,0cyclictest0-21swapper/122:10:411
1315199200,0cyclictest0-21swapper/121:10:431
1315199200,0cyclictest0-21swapper/120:05:401
13148992020,0cyclictest0-21swapper/023:25:410
13148992020,0cyclictest0-21swapper/023:15:430
13148992020,0cyclictest0-21swapper/023:10:380
13148992020,0cyclictest0-21swapper/023:05:410
13148992020,0cyclictest0-21swapper/021:50:000
13148992020,0cyclictest0-21swapper/020:20:410
13148992019,0cyclictest0-21swapper/023:55:420
1314899200,20cyclictest0-21swapper/000:35:450
1314899200,20cyclictest0-21swapper/000:00:430
1314899200,0cyclictest0-21swapper/019:15:400
13158991919,0cyclictest4938-21apt-get00:25:433
13158991919,0cyclictest0-21swapper/321:25:413
13158991919,0cyclictest0-21swapper/320:35:403
13158991918,1cyclictest0-21swapper/320:20:413
1315899190,1cyclictest0-21swapper/319:30:393
1315899190,0cyclictest0-21swapper/322:20:413
1315899190,0cyclictest0-21swapper/320:45:443
1315899190,0cyclictest0-21swapper/320:25:413
13154991919,0cyclictest0-21swapper/222:35:402
13154991919,0cyclictest0-21swapper/222:25:412
13154991919,0cyclictest0-21swapper/221:15:402
13154991919,0cyclictest0-21swapper/221:05:392
13154991919,0cyclictest0-21swapper/200:35:452
13154991919,0cyclictest0-21swapper/200:25:442
1315499190,19cyclictest0-21swapper/219:15:402
13151991919,0cyclictest0-21swapper/123:55:421
13151991919,0cyclictest0-21swapper/121:35:401
13151991919,0cyclictest0-21swapper/100:35:441
13148991919,0cyclictest0-21swapper/023:40:430
13148991919,0cyclictest0-21swapper/019:55:410
13148991919,0cyclictest0-21swapper/019:35:390
13148991919,0cyclictest0-21swapper/019:25:390
13148991919,0cyclictest0-21swapper/000:20:450
13148991919,0cyclictest0-21swapper/000:10:410
13148991919,0cyclictest0-21swapper/000:05:420
13148991918,0cyclictest0-21swapper/019:20:420
13148991918,0cyclictest0-21swapper/000:15:430
13148991917,1cyclictest8461-21turbostat22:20:010
13158991818,0cyclictest0-21swapper/319:45:393
13158991817,1cyclictest7865-21apt-get20:05:403
13158991816,1cyclictest5174-21sendmail-msp20:00:013
13158991816,1cyclictest2891-21cron23:15:013
13154991818,0cyclictest2546-21apt-get00:20:452
13154991818,0cyclictest13583-21apt-get22:30:402
13154991818,0cyclictest0-21swapper/222:00:392
13154991818,0cyclictest0-21swapper/221:55:402
13154991818,0cyclictest0-21swapper/221:10:422
13154991818,0cyclictest0-21swapper/220:10:412
13154991818,0cyclictest0-21swapper/200:30:412
13154991818,0cyclictest0-21swapper/200:00:432
13154991817,1cyclictest15253-21apt-get23:40:432
13154991816,1cyclictest5346-21sendmail-msp23:20:012
13151991818,0cyclictest0-21swapper/122:05:431
13151991818,0cyclictest0-21swapper/121:45:001
13151991818,0cyclictest0-21swapper/121:00:411
13151991818,0cyclictest0-21swapper/119:50:401
13151991818,0cyclictest0-21swapper/119:25:391
13151991817,1cyclictest29732-21apt-get00:10:411
1315199180,2cyclictest22036-21munin-run20:35:011
13148991818,0cyclictest0-21swapper/021:00:410
13148991817,1cyclictest0-21swapper/022:10:010
13158991717,0cyclictest10428-21apt-get23:30:403
13158991717,0cyclictest0-21swapper/322:00:393
13158991717,0cyclictest0-21swapper/320:00:413
13154991717,0cyclictest27875-21apt-get20:45:432
13154991717,0cyclictest17648-21apt-get23:45:422
13154991717,0cyclictest0-21swapper/223:50:412
13154991717,0cyclictest0-21swapper/223:50:412
13154991717,0cyclictest0-21swapper/222:15:412
13154991717,0cyclictest0-21swapper/222:05:432
13154991717,0cyclictest0-21swapper/219:55:412
13154991717,0cyclictest0-21swapper/219:45:402
13151991717,0cyclictest0-21swapper/123:40:431
13151991717,0cyclictest0-21swapper/119:15:411
13151991716,0cyclictest25082-21apt-get20:40:411
13151991716,0cyclictest0-21swapper/100:00:431
13148991717,0cyclictest0-21swapper/022:55:400
13148991717,0cyclictest0-21swapper/020:55:400
13148991716,1cyclictest23292-21perf22:50:000
13158991616,0cyclictest25082-21apt-get20:40:403
13158991616,0cyclictest0-21swapper/322:55:413
13158991616,0cyclictest0-21swapper/322:30:403
13158991616,0cyclictest0-21swapper/321:50:433
13158991614,1cyclictest5115-21apt-get21:05:383
13158991614,1cyclictest31830-21perf00:15:003
13158991612,2cyclictest0-21swapper/319:10:413
1315899160,1cyclictest20722-21grep19:25:003
13154991616,0cyclictest0-21swapper/221:35:402
13154991616,0cyclictest0-21swapper/219:25:392
13154991616,0cyclictest0-21swapper/200:10:412
13154991616,0cyclictest0-21swapper/200:10:012
1315199163,12cyclictest611-21irqbalance21:45:321
13151991616,0cyclictest0-21swapper/122:20:411
13151991616,0cyclictest0-21swapper/119:40:401
13151991614,1cyclictest9426-21grep00:35:011
13148991616,0cyclictest0-21swapper/023:00:420
13148991616,0cyclictest0-21swapper/022:10:410
13148991616,0cyclictest0-21swapper/022:10:410
13148991616,0cyclictest0-21swapper/021:35:400
13148991616,0cyclictest0-21swapper/020:15:380
13148991616,0cyclictest0-21swapper/020:05:400
13148991615,1cyclictest14728-21apt-get21:25:400
13148991615,1cyclictest0-21swapper/019:30:380
13148991614,1cyclictest622-21gdbus21:20:010
13148991614,1cyclictest0-21swapper/023:49:410
1314899160,4cyclictest0-21swapper/021:20:410
13158991515,0cyclictest0-21swapper/300:05:423
13158991513,1cyclictest4272-21ntp_states23:15:483
13154991515,0cyclictest0-21swapper/223:05:422
13154991515,0cyclictest0-21swapper/221:20:422
13154991515,0cyclictest0-21swapper/220:40:412
13154991515,0cyclictest0-21swapper/220:20:422
13154991515,0cyclictest0-21swapper/219:35:392
1315499150,1cyclictest22813-21grep22:45:512
13151991515,0cyclictest0-21swapper/123:25:411
13151991515,0cyclictest0-21swapper/120:15:381
13151991515,0cyclictest0-21swapper/119:20:411
13151991515,0cyclictest0-21swapper/100:25:431
13151991513,1cyclictest7712-21mailstats22:15:491
1315199150,0cyclictest0-21swapper/121:51:321
13148991515,0cyclictest0-21swapper/023:30:400
13148991515,0cyclictest0-21swapper/020:40:410
1314899150,14cyclictest0-21swapper/022:20:410
13158991414,0cyclictest0-21swapper/323:45:413
13158991414,0cyclictest0-21swapper/300:15:433
1315899140,14cyclictest0-21swapper/323:50:373
1315899140,14cyclictest0-21swapper/323:50:373
1315899140,14cyclictest0-21swapper/322:25:013
1315899140,0cyclictest0-21swapper/321:20:543
13154991414,0cyclictest0-21swapper/222:12:332
13154991414,0cyclictest0-21swapper/222:12:332
13154991414,0cyclictest0-21swapper/219:30:152
1315499140,14cyclictest0-21swapper/221:27:532
1315499140,14cyclictest0-21swapper/221:00:162
1315499140,14cyclictest0-21swapper/220:25:392
1315499140,14cyclictest0-21swapper/219:20:422
1315499140,0cyclictest0-21swapper/223:35:252
13151991414,0cyclictest0-21swapper/123:15:161
13151991414,0cyclictest0-21swapper/119:45:301
1315199140,14cyclictest0-21swapper/120:00:351
1315199140,0cyclictest0-21swapper/123:05:231
1315199140,0cyclictest0-21swapper/122:56:091
13148991414,0cyclictest0-21swapper/019:50:360
13148991414,0cyclictest0-21swapper/019:47:370
1314899140,14cyclictest0-21swapper/022:35:180
1314899140,14cyclictest0-21swapper/021:05:090
1314899140,14cyclictest0-21swapper/020:50:210
1314899140,0cyclictest0-21swapper/022:00:380
1314899140,0cyclictest0-21swapper/020:10:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional