You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 17:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Mon Dec 08, 2025 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1606599250,1cyclictest593-21dbus-daemon11:40:001
16063992523,1cyclictest0-21swapper/012:10:000
1607599240,23cyclictest13976-21grep10:25:003
16071992421,2cyclictest0-21swapper/208:05:012
1607199240,23cyclictest0-21swapper/207:38:342
16063992424,0cyclictest0-21swapper/011:43:350
1606399240,23cyclictest3439-21apt-get12:18:350
1606399240,23cyclictest15798-21munin-plugin-st08:15:000
1607599230,0cyclictest0-21swapper/307:13:343
1607199230,22cyclictest0-21swapper/208:13:342
1606599230,22cyclictest609-21polkitd07:39:591
1606599230,22cyclictest28393-21awk10:55:001
16063992321,1cyclictest0-21swapper/012:35:010
16063992320,2cyclictest23598-21dump-pmu-power10:44:590
1606399231,21cyclictest22855-21cron11:50:000
1606399230,2cyclictest22277-21awk09:35:000
1606399230,22cyclictest5724-21perf07:55:000
1606399230,22cyclictest19485-21apt-get09:30:000
1606399230,22cyclictest13397-21munin-plugin-st08:10:000
16075992222,0cyclictest31003-21apt-get12:08:363
16075992220,1cyclictest921-21latency_hist07:45:013
16075992220,1cyclictest8311-21apt-get12:28:383
16075992220,1cyclictest30895-21cron07:40:003
16075992220,1cyclictest0-21swapper/308:55:003
1607599220,2cyclictest15364-21latency_hist12:40:003
1607599220,21cyclictest612-21gdbus10:45:003
1607599220,21cyclictest0-21swapper/309:48:363
1607599220,21cyclictest0-21swapper/307:58:353
1607599220,21cyclictest0-21swapper/307:30:023
16071992221,0cyclictest23536-21apt-get08:33:332
16071992220,1cyclictest7770-21awk12:25:002
16071992220,1cyclictest21177-21perf10:40:002
1607199221,0cyclictest271rcuc/208:23:352
1607199220,21cyclictest8211-21munin-run08:00:002
1607199220,21cyclictest3328-21perf07:50:002
1607199220,21cyclictest20426-21apt-get09:33:362
1607199220,21cyclictest0-21swapper/212:13:372
1607199220,21cyclictest0-21swapper/211:48:352
1607199220,21cyclictest0-21swapper/211:18:372
1607199220,21cyclictest0-21swapper/208:30:012
1607199220,1cyclictest31913-21munin-run09:55:002
16065992222,0cyclictest0-21swapper/109:13:341
16065992221,0cyclictest19304-21apt-get10:38:371
16065992219,2cyclictest25981-21dump-pmu-power10:50:001
1606599220,21cyclictest8456-21sendmail-msp11:20:001
1606599220,21cyclictest5026-21perf09:00:001
1606599220,21cyclictest0-21swapper/111:35:001
1606599220,21cyclictest0-21swapper/111:28:351
1606599220,1cyclictest20622-21munin-run08:25:001
1606599220,1cyclictest13920-21apt-get08:13:341
1606599220,0cyclictest0-21swapper/107:43:341
1606399220,21cyclictest4975-21cron09:00:000
1606399220,21cyclictest28607-21apt-get12:03:360
1606399220,21cyclictest18946-21apt-get07:18:330
1606399220,1cyclictest26248-21apt-get07:30:020
16075992121,0cyclictest0-21swapper/311:38:353
16075992120,1cyclictest11101-21apt-get12:33:363
16075992119,1cyclictest13229-21dump-pmu-power11:30:003
16075992118,2cyclictest10570-21cron08:05:003
1607599210,21cyclictest0-21swapper/312:23:353
1607599210,21cyclictest0-21swapper/312:03:353
1607599210,20cyclictest0-21swapper/310:13:363
1607599210,20cyclictest0-21swapper/308:13:333
1607599210,1cyclictest0-21swapper/308:20:003
1607599210,0cyclictest30748-21apt-get08:48:343
1607599210,0cyclictest21367-21apt-get07:23:343
1607599210,0cyclictest11526-21apt-get08:08:333
1607599210,0cyclictest0-21swapper/311:13:363
16071992121,0cyclictest0-21swapper/209:48:362
16071992119,1cyclictest0-21swapper/212:18:352
16071992117,3cyclictest31103-21apt-config11:00:002
1607199210,21cyclictest0-21swapper/207:13:332
1607199210,20cyclictest23577-21sh10:45:002
1607199210,20cyclictest15224-21apt-get09:23:362
1607199210,20cyclictest0-21swapper/211:58:362
1607199210,20cyclictest0-21swapper/211:43:352
1607199210,20cyclictest0-21swapper/211:23:352
1607199210,20cyclictest0-21swapper/207:23:342
1607199210,1cyclictest0-21swapper/211:00:012
1607199210,1cyclictest0-21swapper/207:18:432
1607199210,0cyclictest32430-21apt-get09:58:352
1607199210,0cyclictest0-21swapper/211:08:352
16065992121,0cyclictest19485-21apt-get09:30:091
16065992121,0cyclictest0-21swapper/112:23:351
16065992121,0cyclictest0-21swapper/109:43:341
16065992119,1cyclictest0-21swapper/112:20:001
1606599210,21cyclictest0-21swapper/112:13:371
1606599210,21cyclictest0-21swapper/110:18:351
1606599210,21cyclictest0-21swapper/107:33:381
1606599210,20cyclictest6317-21apt-get07:58:351
1606599210,20cyclictest0-21swapper/111:43:351
1606599210,0cyclictest8981-21apt-get11:23:351
1606599210,0cyclictest1711-21apt-get11:08:361
1606599210,0cyclictest0-21swapper/108:53:351
16063992121,0cyclictest0-21swapper/009:03:360
16063992121,0cyclictest0-21swapper/008:53:340
16063992118,3cyclictest0-21swapper/010:13:360
1606399210,20cyclictest12102-21apt-get10:23:340
1606399210,20cyclictest0-21swapper/011:08:360
1606399210,20cyclictest0-21swapper/010:53:340
1606399210,20cyclictest0-21swapper/008:33:330
1606399210,0cyclictest4836-21apt-get10:08:350
1606399210,0cyclictest28953-21apt-get07:38:340
1606399210,0cyclictest25936-21apt-get08:38:350
1606399210,0cyclictest23764-21apt-get07:28:340
1606399210,0cyclictest1450-21apt-get07:48:320
16075992020,0cyclictest18745-21apt-get08:23:353
16075992020,0cyclictest0-21swapper/310:08:353
16075992020,0cyclictest0-21swapper/309:53:353
16075992020,0cyclictest0-21swapper/309:28:353
16075992020,0cyclictest0-21swapper/308:43:323
16075992020,0cyclictest0-21swapper/307:28:343
16075992018,1cyclictest5279-21cron12:20:003
16075992018,1cyclictest0-21swapper/310:33:363
16075992018,1cyclictest0-21swapper/309:30:093
16075992016,3cyclictest27805-21awk08:39:593
1607599200,0cyclictest0-21swapper/311:00:013
16071992020,0cyclictest0-21swapper/211:50:012
16071992020,0cyclictest0-21swapper/210:33:352
16071992020,0cyclictest0-21swapper/210:13:352
16071992020,0cyclictest0-21swapper/209:13:342
16071992020,0cyclictest0-21swapper/208:58:322
16071992019,1cyclictest0-21swapper/211:38:352
16071992019,0cyclictest4110-21apt-get11:13:362
16071992018,1cyclictest30486-21munin-run12:05:002
16071992017,2cyclictest15339-21cron12:40:002
1607199200,19cyclictest0-21swapper/208:53:342
1607199200,0cyclictest0-21swapper/212:28:382
1607199200,0cyclictest0-21swapper/210:28:352
1607199200,0cyclictest0-21swapper/209:03:362
1607199200,0cyclictest0-21swapper/208:48:342
16065992020,0cyclictest0-21swapper/111:53:351
16065992020,0cyclictest0-21swapper/110:08:351
16065992020,0cyclictest0-21swapper/110:03:341
16065992020,0cyclictest0-21swapper/108:35:011
16065992020,0cyclictest0-21swapper/108:28:351
16065992020,0cyclictest0-21swapper/108:18:341
16065992019,1cyclictest0-21swapper/112:10:001
16065992018,1cyclictest593-21dbus-daemon12:05:001
1606599200,0cyclictest0-21swapper/108:48:341
16063992020,0cyclictest11380-21apt-get11:28:350
16063992020,0cyclictest0-21swapper/012:28:370
16063992020,0cyclictest0-21swapper/011:23:350
16063992020,0cyclictest0-21swapper/008:28:370
16063992020,0cyclictest0-21swapper/007:58:340
16063992020,0cyclictest0-21swapper/007:43:340
16063992019,1cyclictest25998-21munin-plugin-st10:50:000
16063992019,1cyclictest2440-21apt-get10:03:350
16063992019,1cyclictest22820-21apt-get09:38:340
16063992019,1cyclictest0-21swapper/012:13:440
16063992018,1cyclictest28064-21grep12:00:000
1606399200,0cyclictest30033-21apt-get09:53:340
1606399200,0cyclictest0-21swapper/008:48:340
16075991919,0cyclictest0-21swapper/311:53:353
16075991919,0cyclictest0-21swapper/310:48:383
16075991919,0cyclictest0-21swapper/309:38:343
16075991917,1cyclictest5034-21grep09:00:003
16075991917,1cyclictest0-21swapper/311:19:593
16071991919,0cyclictest0-21swapper/212:08:362
16071991919,0cyclictest0-21swapper/209:28:352
16071991919,0cyclictest0-21swapper/208:38:352
16071991919,0cyclictest0-21swapper/207:28:352
16071991917,2cyclictest0-21swapper/210:08:382
16071991916,2cyclictest9348-21grep08:03:362
16071991916,2cyclictest27162-21munin-run09:45:002
16071991916,2cyclictest0-21swapper/209:40:002
1607199191,1cyclictest0-21swapper/209:08:412
16065991919,0cyclictest3861-21apt-get07:53:331
16065991919,0cyclictest0-21swapper/112:33:361
16065991919,0cyclictest0-21swapper/111:48:331
16065991919,0cyclictest0-21swapper/110:13:361
16065991919,0cyclictest0-21swapper/107:48:331
16065991919,0cyclictest0-21swapper/107:28:351
16065991918,1cyclictest8006-21apt-get09:08:331
16063991919,0cyclictest0-21swapper/009:08:340
16063991919,0cyclictest0-21swapper/008:43:320
16063991919,0cyclictest0-21swapper/007:13:340
16063991917,1cyclictest6016-21turbostat.cron11:15:000
16063991917,1cyclictest11579-21cut10:20:000
16075991818,0cyclictest18599-21apt-get11:43:343
16075991818,0cyclictest0-21swapper/311:08:363
16075991818,0cyclictest0-21swapper/308:28:373
16075991816,1cyclictest0-21swapper/307:20:003
1607599180,13cyclictest0-21swapper/311:23:353
16071991818,0cyclictest26515-21apt-get10:53:342
16071991817,1cyclictest0-21swapper/212:33:362
16071991816,1cyclictest0-21swapper/208:43:362
16071991815,2cyclictest14143-21cut10:25:002
16071991814,3cyclictest17313-21grep08:18:382
16071991811,6cyclictest0-21swapper/210:48:412
16065991818,0cyclictest14499-21apt-get10:28:351
16065991818,0cyclictest0-21swapper/112:38:371
16065991818,0cyclictest0-21swapper/110:43:361
16065991818,0cyclictest0-21swapper/110:23:341
16065991818,0cyclictest0-21swapper/107:23:341
16065991816,1cyclictest10179-21perf12:30:011
1606599180,0cyclictest28349-21apt-get08:43:321
16063991818,0cyclictest6567-21apt-get11:18:370
16063991818,0cyclictest10406-21apt-get09:13:340
16075991717,0cyclictest0-21swapper/309:23:363
16075991717,0cyclictest0-21swapper/309:08:343
16075991717,0cyclictest0-21swapper/308:33:333
16075991715,1cyclictest0-21swapper/310:05:013
16071991715,1cyclictest5319-21threads07:53:412
16071991713,3cyclictest0-21swapper/209:18:412
16071991712,4cyclictest26152-21dpkg07:30:012
16065991717,0cyclictest27636-21apt-get09:48:371
16065991717,0cyclictest16907-21apt-get10:33:351
16065991717,0cyclictest0-21swapper/111:58:351
16065991717,0cyclictest0-21swapper/111:13:361
16065991717,0cyclictest0-21swapper/109:38:341
16065991717,0cyclictest0-21swapper/107:18:331
16065991716,0cyclictest0-21swapper/109:53:351
16063991717,0cyclictest0-21swapper/010:38:370
16063991717,0cyclictest0-21swapper/009:58:350
16063991717,0cyclictest0-21swapper/009:43:350
1606399170,16cyclictest27604-21/usr/sbin/munin09:48:430
16075991616,0cyclictest0-21swapper/311:48:353
16075991614,1cyclictest1896-21apt10:00:003
16075991614,1cyclictest12806-21apt-get09:18:363
16071991613,2cyclictest0-21swapper/210:18:362
16065991616,0cyclictest0-21swapper/111:03:351
16065991616,0cyclictest0-21swapper/109:58:351
16065991615,0cyclictest0-21swapper/110:58:351
16065991614,1cyclictest5555-21apt-get09:03:351
1606599160,15cyclictest0-21swapper/109:18:371
16063991616,0cyclictest0-21swapper/011:38:340
16063991616,0cyclictest0-21swapper/009:18:360
16063991615,0cyclictest28915-21apt-get10:58:360
16063991614,1cyclictest10585-21sh08:05:000
1606399160,8cyclictest0-21swapper/010:33:350
16075991515,0cyclictest3861-21apt-get07:53:333
16075991515,0cyclictest25237-21apt-get09:43:343
16075991515,0cyclictest0-21swapper/310:58:363
16075991515,0cyclictest0-21swapper/310:53:353
16075991514,1cyclictest9686-21apt-get10:18:353
16075991514,1cyclictest0-21swapper/312:13:433
16071991515,0cyclictest0-21swapper/211:28:352
16071991515,0cyclictest0-21swapper/210:03:452
16063991515,0cyclictest0-21swapper/012:23:360
16063991513,1cyclictest0-21swapper/010:30:000
16075991413,1cyclictest28051-21idleruntime-cro12:00:003
1607599140,14cyclictest0-21swapper/311:31:403
1607599140,14cyclictest0-21swapper/310:36:273
1607599140,14cyclictest0-21swapper/310:27:143
1607599140,14cyclictest0-21swapper/309:13:413
1607599140,0cyclictest0-21swapper/309:01:563
1607599140,0cyclictest0-21swapper/307:45:263
1607199140,14cyclictest0-21swapper/207:42:092
1607199140,0cyclictest0-21swapper/211:32:502
16065991414,0cyclictest0-21swapper/108:05:121
16065991414,0cyclictest0-21swapper/108:02:291
1606599140,14cyclictest0-21swapper/107:11:021
1606599140,0cyclictest0-21swapper/109:25:561
1606599140,0cyclictest0-21swapper/109:20:291
16063991414,0cyclictest0-21swapper/011:00:360
16063991414,0cyclictest0-21swapper/009:21:000
1606399140,14cyclictest0-21swapper/012:35:430
1606399140,14cyclictest0-21swapper/011:32:020
1606399140,14cyclictest0-21swapper/008:20:500
1606399140,14cyclictest0-21swapper/008:15:530
1606399140,14cyclictest0-21swapper/007:20:390
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional