You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-10 - 07:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed Dec 10, 2025 00:44:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16363992622,4cyclictest0-21swapper/123:48:431
16365992523,1cyclictest28362-21latency_hist19:35:003
1636399250,24cyclictest11229-21apt-get23:28:451
1636499240,1cyclictest0-21swapper/223:48:432
16363992422,1cyclictest0-21swapper/120:05:001
16363992421,2cyclictest9666-21awk21:10:001
16363992420,3cyclictest28903-21apt-get19:38:441
1636399240,23cyclictest7218-21awk00:25:001
1636399240,1cyclictest27257-21perf20:40:001
1636399240,1cyclictest11949-21apt-get22:23:431
1636399240,1cyclictest11949-21apt-get22:23:431
1636299240,24cyclictest0-21swapper/000:18:430
16365992323,0cyclictest13362-21apt-get20:13:403
1636599230,22cyclictest21141-21apt19:20:013
16364992321,1cyclictest598-21rs:main2
1636499230,22cyclictest30779-21munin-run19:40:002
1636499230,0cyclictest0-21swapper/222:23:432
1636499230,0cyclictest0-21swapper/222:23:422
16363992321,1cyclictest30635-21munin-run23:00:001
16363992321,1cyclictest20319-21munin-plugin-st23:45:001
1636399230,1cyclictest28209-21latency_hist22:55:001
1636399230,1cyclictest16937-21ls21:25:001
16362992323,0cyclictest0-21swapper/020:03:400
16362992321,1cyclictest0-21swapper/019:45:000
1636299230,2cyclictest30599-21cron23:00:000
1636299230,22cyclictest563-21apt23:04:590
1636299230,22cyclictest0-21swapper/000:30:000
16365992222,0cyclictest0-21swapper/321:08:413
16365992220,1cyclictest3703-21apt-get19:53:413
16365992220,1cyclictest27483-40mandb00:00:003
16365992220,1cyclictest27483-40mandb00:00:003
16365992220,1cyclictest21019-21perf22:40:013
16365992220,1cyclictest16023-21apt-get23:38:433
16365992220,1cyclictest13091-21awk23:30:013
16365992220,1cyclictest12835-21idleruntime-cro20:10:013
16365992220,1cyclictest11405-21perf22:20:003
1636599220,21cyclictest0-21swapper/322:28:433
1636599220,21cyclictest0-21swapper/321:48:433
1636599220,21cyclictest0-21swapper/321:18:443
16364992222,0cyclictest7078-21apt-get22:13:412
16364992222,0cyclictest0-21swapper/219:48:432
16364992220,1cyclictest3958-21apt-get23:13:422
16364992220,1cyclictest3958-21apt-get23:13:422
16364992220,1cyclictest23416-21grep22:45:002
16364992220,1cyclictest16943-21perf21:24:592
16364992219,2cyclictest11992-21wc00:35:002
1636499220,21cyclictest25085-21apt-get21:43:442
1636499220,21cyclictest1705-21sh22:00:002
1636499220,21cyclictest0-21swapper/223:23:412
1636499220,21cyclictest0-21swapper/223:05:022
1636499220,21cyclictest0-21swapper/219:53:422
1636499220,21cyclictest0-21swapper/200:13:432
1636499220,1cyclictest22990-21apt-get20:33:442
16363992220,1cyclictest15464-21awk23:35:001
16363992217,4cyclictest19319-21turbostat.cron21:30:001
1636399221,1cyclictest14504-21cron21:20:001
1636399220,21cyclictest8776-21apt-get23:23:421
1636399220,21cyclictest0-21swapper/120:58:441
1636399220,21cyclictest0-21swapper/100:18:431
16362992222,0cyclictest20849-21apt-get23:48:430
16362992222,0cyclictest0-21swapper/021:23:450
16362992220,1cyclictest23429-21munin-run22:45:000
16362992219,2cyclictest18594-21awk22:35:000
1636299220,21cyclictest28150-21apt-get00:03:430
1636299220,21cyclictest14409-21perf00:39:590
1636299220,21cyclictest1294-21apt-get19:48:430
1636299220,21cyclictest0-21swapper/021:58:450
1636299220,21cyclictest0-21swapper/021:03:410
1636299220,1cyclictest612-21gdbus20:15:010
16365992121,0cyclictest0-21swapper/321:23:453
16365992121,0cyclictest0-21swapper/320:28:413
16365992120,1cyclictest25385-21apt-get20:38:433
16365992119,1cyclictest0-21swapper/322:50:003
16365992119,1cyclictest0-21swapper/321:40:023
1636599210,21cyclictest0-21swapper/323:33:423
1636599210,20cyclictest0-21swapper/322:33:433
1636599210,20cyclictest0-21swapper/322:03:423
1636599210,20cyclictest0-21swapper/320:43:453
1636599210,1cyclictest0-21swapper/321:15:003
1636599210,0cyclictest21708-21apt-get19:23:423
1636599210,0cyclictest0-21swapper/319:48:433
1636599210,0cyclictest0-21swapper/300:18:433
16364992119,1cyclictest31701-21cron21:55:002
16364992119,1cyclictest21116-21cron19:20:012
16364992119,1cyclictest0-21swapper/200:00:002
16364992119,1cyclictest0-21swapper/200:00:002
1636499210,20cyclictest0-21swapper/200:28:462
1636499210,1cyclictest32396-21latency_hist00:10:002
1636499210,1cyclictest26364-21apt-get22:53:432
16363992120,0cyclictest0-21swapper/123:08:451
16363992120,0cyclictest0-21swapper/122:03:421
16363992119,1cyclictest0-21swapper/120:18:421
1636399210,20cyclictest0-21swapper/119:43:411
1636399210,1cyclictest13362-21apt-get20:13:401
1636399210,0cyclictest4686-21apt-get22:08:421
1636399210,0cyclictest0-21swapper/121:13:431
1636399210,0cyclictest0-21swapper/120:48:461
1636399210,0cyclictest0-21swapper/120:23:411
16362992121,0cyclictest0-21swapper/019:23:430
16362992119,1cyclictest0-21swapper/021:35:000
1636299210,21cyclictest0-21swapper/022:18:420
1636299210,21cyclictest0-21swapper/022:03:430
1636299210,1cyclictest10970-21apt-get20:08:410
1636299210,0cyclictest10200-21apt-get21:13:420
1636299210,0cyclictest0-21swapper/021:40:020
16365992020,0cyclictest0-21swapper/323:18:423
16365992020,0cyclictest0-21swapper/322:23:433
16365992020,0cyclictest0-21swapper/322:23:433
16365992020,0cyclictest0-21swapper/320:03:423
16365992018,1cyclictest0-21swapper/320:50:013
1636599200,20cyclictest0-21swapper/320:23:423
1636599200,1cyclictest14420-21munin-run00:40:003
16364992019,1cyclictest0-21swapper/222:58:442
16364992019,0cyclictest8518-21apt-get20:03:412
16364992019,0cyclictest0-21swapper/220:08:422
1636499200,20cyclictest0-21swapper/220:53:432
1636499200,19cyclictest0-21swapper/223:38:432
1636499200,0cyclictest0-21swapper/222:38:432
1636499200,0cyclictest0-21swapper/222:33:432
16363992020,0cyclictest9478-21apt-get22:18:421
16363992020,0cyclictest0-21swapper/123:13:411
16363992020,0cyclictest0-21swapper/123:13:411
16363992020,0cyclictest0-21swapper/122:48:421
16363992020,0cyclictest0-21swapper/122:43:421
16363992020,0cyclictest0-21swapper/121:38:421
16363992020,0cyclictest0-21swapper/120:08:421
16363992020,0cyclictest0-21swapper/119:53:411
16363992020,0cyclictest0-21swapper/119:13:421
16363992019,1cyclictest10143-21apt-get00:33:431
1636399200,1cyclictest7602-21perl22:13:421
1636399200,0cyclictest0-21swapper/121:58:441
1636399200,0cyclictest0-21swapper/100:13:431
16362992020,0cyclictest5353-21apt-get00:23:430
16362992020,0cyclictest0-21swapper/023:58:430
16362992020,0cyclictest0-21swapper/023:58:420
16362992020,0cyclictest0-21swapper/023:08:450
16362992020,0cyclictest0-21swapper/021:28:440
16362992020,0cyclictest0-21swapper/020:18:420
16362992020,0cyclictest0-21swapper/019:33:430
16362992020,0cyclictest0-21swapper/000:33:420
16362992019,1cyclictest0-21swapper/023:18:420
16362992019,0cyclictest18191-21apt-get20:23:420
16362992018,1cyclictest30773-21perf19:40:000
16362992018,1cyclictest0-21swapper/020:55:000
1636299200,0cyclictest0-21swapper/023:38:430
16365991919,0cyclictest0-21swapper/322:08:443
16365991919,0cyclictest0-21swapper/319:28:443
16365991917,1cyclictest32367-21cron00:10:003
16364991919,0cyclictest0-21swapper/220:13:402
16364991919,0cyclictest0-21swapper/219:35:002
16364991919,0cyclictest0-21swapper/219:28:432
16364991918,1cyclictest0-21swapper/222:08:432
16364991917,1cyclictest313-21systemd-journal20:50:002
16364991917,1cyclictest2285-21apt-get22:03:432
16364991916,2cyclictest9623-21cron21:10:002
1636499190,0cyclictest0-21swapper/221:48:432
16363991919,0cyclictest0-21swapper/123:58:441
16363991919,0cyclictest0-21swapper/123:58:441
16363991919,0cyclictest0-21swapper/119:23:431
16363991919,0cyclictest0-21swapper/100:08:441
16363991917,1cyclictest313-21systemd-journal20:35:001
1636399190,1cyclictest6765-21mailstats21:03:491
16362991919,0cyclictest30546-21apt-get00:08:440
16365991818,0cyclictest0-21swapper/323:08:453
16365991818,0cyclictest0-21swapper/322:53:433
16365991818,0cyclictest0-21swapper/322:43:413
16365991818,0cyclictest0-21swapper/321:33:433
16365991818,0cyclictest0-21swapper/320:48:463
16365991817,1cyclictest18452-21apt-get23:43:453
16364991818,0cyclictest5399-21apt-get21:03:412
16364991818,0cyclictest2934-21apt-get00:18:432
16364991818,0cyclictest12544-21apt-get00:38:472
16364991818,0cyclictest0-21swapper/223:03:442
16364991818,0cyclictest0-21swapper/220:18:422
1636499180,10cyclictest0-21swapper/220:23:422
16363991818,0cyclictest27483-21apt-get21:48:431
16363991818,0cyclictest0-21swapper/123:53:441
16363991818,0cyclictest0-21swapper/122:33:431
16363991818,0cyclictest0-21swapper/121:33:431
16363991818,0cyclictest0-21swapper/120:28:401
16363991818,0cyclictest0-21swapper/119:33:431
16363991816,1cyclictest16190-21awk22:30:001
1636399180,0cyclictest0-21swapper/119:48:441
16362991818,0cyclictest19289-21apt-get19:18:420
16362991818,0cyclictest0-21swapper/022:53:420
16362991818,0cyclictest0-21swapper/021:48:430
16362991818,0cyclictest0-21swapper/020:58:440
16362991818,0cyclictest0-21swapper/019:58:420
16362991818,0cyclictest0-21swapper/019:53:420
16362991818,0cyclictest0-21swapper/019:13:430
16362991816,1cyclictest22451-21munin-run20:30:000
16365991717,0cyclictest31318-21apt-get19:43:413
16365991717,0cyclictest0-21swapper/323:48:433
16365991717,0cyclictest0-21swapper/323:13:423
16365991717,0cyclictest0-21swapper/323:13:423
16365991717,0cyclictest0-21swapper/321:03:423
16365991717,0cyclictest0-21swapper/320:33:453
16365991717,0cyclictest0-21swapper/320:18:423
16365991717,0cyclictest0-21swapper/319:58:423
16365991717,0cyclictest0-21swapper/300:23:423
16364991717,0cyclictest0-21swapper/223:53:442
16364991717,0cyclictest0-21swapper/222:18:422
16364991717,0cyclictest0-21swapper/221:28:452
16364991717,0cyclictest0-21swapper/220:38:422
16364991717,0cyclictest0-21swapper/219:43:422
16363991717,0cyclictest7751-21apt-get00:28:461
16363991717,0cyclictest0-21swapper/121:40:021
16363991717,0cyclictest0-21swapper/100:03:421
16362991717,0cyclictest11949-21apt-get22:23:420
16362991717,0cyclictest11949-21apt-get22:23:420
16362991717,0cyclictest0-21swapper/023:43:450
16362991717,0cyclictest0-21swapper/023:33:430
16362991717,0cyclictest0-21swapper/023:28:440
16362991717,0cyclictest0-21swapper/022:48:420
16362991717,0cyclictest0-21swapper/021:53:420
16362991717,0cyclictest0-21swapper/021:18:440
16365991616,0cyclictest0-21swapper/321:38:423
16365991615,1cyclictest23248-21apt-get23:53:443
16365991614,1cyclictest0-21swapper/319:15:003
16365991614,1cyclictest0-21swapper/300:15:013
16364991616,0cyclictest0-21swapper/223:33:432
16364991616,0cyclictest0-21swapper/220:58:442
16364991616,0cyclictest0-21swapper/220:43:452
16364991615,0cyclictest19878-21apt-get21:33:432
16364991614,1cyclictest23546-21ls19:24:592
16363991614,1cyclictest24742-21sed19:28:461
1636399160,1cyclictest0-21swapper/122:38:491
16362991616,0cyclictest7796-21apt-get21:08:410
16362991616,0cyclictest0-21swapper/022:38:430
16362991616,0cyclictest0-21swapper/022:28:440
16362991614,1cyclictest0-21swapper/020:43:540
1636299160,15cyclictest26508-21phc2sys-jitter20:38:500
1636299160,15cyclictest0-21swapper/021:40:000
16365991515,0cyclictest0-21swapper/322:13:413
16365991515,0cyclictest0-21swapper/321:28:443
16364991515,0cyclictest0-21swapper/222:48:422
16364991514,0cyclictest6358-21apt-get23:18:422
16363991515,0cyclictest6100-21apt-get19:58:411
16363991515,0cyclictest29879-21apt-get21:53:421
16363991515,0cyclictest0-21swapper/123:38:431
16363991514,1cyclictest0-21swapper/123:04:221
16362991515,0cyclictest0-21swapper/023:13:420
16362991515,0cyclictest0-21swapper/023:13:420
16365991414,0cyclictest0-21swapper/321:56:143
16365991414,0cyclictest0-21swapper/300:33:433
1636599140,14cyclictest0-21swapper/323:00:403
1636599140,14cyclictest0-21swapper/321:53:513
1636599140,14cyclictest0-21swapper/320:56:223
1636599140,14cyclictest0-21swapper/319:35:423
1636599140,14cyclictest0-21swapper/300:26:193
1636499140,14cyclictest0-21swapper/223:44:502
1636499140,14cyclictest0-21swapper/223:26:172
1636499140,14cyclictest0-21swapper/219:56:132
1636499140,14cyclictest0-21swapper/219:10:432
1636499140,14cyclictest0-21swapper/200:00:172
1636499140,0cyclictest0-21swapper/221:15:102
1636499140,0cyclictest0-21swapper/220:25:462
1636499140,0cyclictest0-21swapper/200:20:472
1636399140,14cyclictest0-21swapper/120:50:491
1636399140,0cyclictest0-21swapper/120:40:091
16362991414,0cyclictest0-21swapper/022:05:130
1636299140,14cyclictest0-21swapper/023:50:120
1636299140,14cyclictest0-21swapper/023:23:420
1636299140,14cyclictest0-21swapper/019:25:560
1636299140,0cyclictest0-21swapper/022:10:210
1636299140,0cyclictest0-21swapper/020:45:430
1636299140,0cyclictest0-21swapper/020:31:040
1636299140,0cyclictest0-21swapper/000:11:030
16365991313,0cyclictest0-21swapper/323:21:393
16365991313,0cyclictest0-21swapper/322:55:423
16364991313,0cyclictest0-21swapper/221:11:092
16363991313,0cyclictest0-21swapper/100:36:401
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional