You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-17 - 21:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Wed Dec 17, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30904992523,1cyclictest18181-21latency_hist07:50:010
30904992520,5cyclictest0-21swapper/010:49:360
30904992520,4cyclictest0-21swapper/011:49:360
30914992421,2cyclictest606-21dbus-daemon10:20:012
3091499240,2cyclictest313-21systemd-journal07:15:012
30904992421,3cyclictest0-21swapper/011:19:370
3090499240,0cyclictest0-21swapper/007:29:350
3091799230,22cyclictest8790-21cron10:50:003
3091799230,22cyclictest0-21swapper/310:45:003
30914992321,1cyclictest19653-21latency_hist12:20:002
30914992321,1cyclictest0-21swapper/207:50:012
3091499230,22cyclictest6620-21apt-get10:45:022
3091499230,22cyclictest11800-21apt-get10:59:372
3090999230,22cyclictest0-21swapper/110:19:351
3090999230,22cyclictest0-21swapper/107:39:371
3090999230,1cyclictest15463-21apt-get12:14:361
30917992221,0cyclictest1798-21apt-get10:39:373
30917992220,1cyclictest6281-21apt-get07:29:353
30917992220,1cyclictest0-21swapper/311:50:013
3091799220,21cyclictest1231-21apt-get08:24:353
3091799220,1cyclictest11486-21latency_hist09:50:003
30914992220,1cyclictest5251-21turbostat11:50:012
3091499220,21cyclictest9404-21apt-get10:54:352
3091499220,21cyclictest8258-21apt-get11:59:342
3091499220,21cyclictest32509-21apt-get09:29:352
3091499220,21cyclictest30108-21apt-get09:24:352
3091499220,21cyclictest0-21swapper/209:04:372
3091499220,1cyclictest0-21swapper/211:20:012
30909992220,1cyclictest16290-21apt-get10:00:021
30909992220,1cyclictest0-21swapper/109:55:001
30909992219,2cyclictest19680-21idleruntime-cro12:20:001
3090999220,21cyclictest8678-21apt-get07:34:351
3090999220,21cyclictest5262-21perf11:50:011
3090999220,21cyclictest26822-21cron12:35:011
3090999220,21cyclictest25240-21apt-get09:14:351
3090999220,21cyclictest0-21swapper/112:09:371
3090999220,21cyclictest0-21swapper/112:04:351
3090999220,21cyclictest0-21swapper/111:19:371
3090999220,21cyclictest0-21swapper/110:49:361
3090999220,21cyclictest0-21swapper/109:39:351
3090999220,21cyclictest0-21swapper/107:49:351
30904992221,1cyclictest3639-21apt-get08:29:360
30904992221,1cyclictest19001-21apt-get11:14:360
30904992220,1cyclictest28762-21munin-run10:25:010
30904992220,1cyclictest24492-21apt-get10:19:350
30904992218,4cyclictest0-21swapper/012:09:370
30904992217,5cyclictest0-21swapper/012:14:360
3090499220,21cyclictest11195-21latency_hist10:55:010
3090499220,21cyclictest0-21swapper/012:04:340
3090499220,21cyclictest0-21swapper/009:49:360
3090499220,21cyclictest0-21swapper/008:59:350
3090499220,1cyclictest26208-21apt-get11:29:360
3090499220,0cyclictest0-21swapper/010:34:360
30917992119,1cyclictest29358-21apt-get10:29:353
30917992119,1cyclictest26307-21munin-run10:20:013
30917992119,1cyclictest0-21swapper/311:44:363
30917992119,1cyclictest0-21swapper/308:35:013
3091799210,20cyclictest0-21swapper/310:59:373
3091799210,20cyclictest0-21swapper/310:34:363
3091799210,20cyclictest0-21swapper/309:09:363
3091799210,1cyclictest0-21swapper/309:34:353
3091799210,0cyclictest0-21swapper/312:39:493
3091799210,0cyclictest0-21swapper/311:59:343
3091799210,0cyclictest0-21swapper/310:54:353
3091799210,0cyclictest0-21swapper/309:29:353
3091799210,0cyclictest0-21swapper/309:24:353
3091799210,0cyclictest0-21swapper/308:29:363
30914992121,0cyclictest0-21swapper/210:39:372
30914992120,0cyclictest26208-21apt-get11:29:352
30914992120,0cyclictest18784-21apt-get07:54:342
30914992119,1cyclictest7805-21cron08:35:012
30914992119,1cyclictest26910-21apt-get10:24:372
3091499210,20cyclictest0-21swapper/208:24:352
3091499210,0cyclictest31765-21apt-get10:34:362
3091499210,0cyclictest0-21swapper/212:39:492
3091499210,0cyclictest0-21swapper/207:29:342
30909992121,0cyclictest27476-21apt-get12:39:491
30909992121,0cyclictest0-21swapper/109:44:351
30909992119,1cyclictest13244-21apt-get08:49:371
30909992119,1cyclictest0-21swapper/111:55:011
30909992119,1cyclictest0-21swapper/111:40:001
3090999210,20cyclictest4216-21apt-get10:44:371
3090999210,0cyclictest0-21swapper/111:59:341
30904992121,0cyclictest0-21swapper/008:39:370
30904992118,2cyclictest15773-21wc07:45:010
3090499210,2cyclictest19624-21seq07:54:390
3090499210,21cyclictest0-21swapper/008:14:370
3090499210,20cyclictest0-21swapper/010:59:370
3090499210,20cyclictest0-21swapper/008:34:360
3090499210,20cyclictest0-21swapper/008:19:350
30917992020,0cyclictest0-21swapper/311:29:353
30917992020,0cyclictest0-21swapper/307:54:343
30917992020,0cyclictest0-21swapper/307:50:013
30917992020,0cyclictest0-21swapper/307:34:353
30917992019,1cyclictest8434-21apt-get08:39:373
30917992019,1cyclictest13071-21apt-get12:09:373
30917992018,1cyclictest20277-21apt-get12:24:363
3091799200,20cyclictest0-21swapper/308:59:353
3091799200,0cyclictest14495-21apt-get09:59:373
3091799200,0cyclictest14214-21apt-get11:04:363
3091799200,0cyclictest14214-21apt-get11:04:363
3091799200,0cyclictest0-21swapper/311:34:353
30914992020,0cyclictest0-21swapper/208:29:362
30914992019,1cyclictest26385-21apt-get08:09:362
30914992019,0cyclictest16606-21apt-get11:09:352
30914992019,0cyclictest0-21swapper/209:34:352
30914992019,0cyclictest0-21swapper/208:04:352
30914992018,1cyclictest1460-21apt-get07:19:362
3091499200,19cyclictest31163-21idleruntime-cro10:30:012
3091499200,19cyclictest0-21swapper/207:34:342
30909992020,0cyclictest0-21swapper/111:24:351
30909992020,0cyclictest0-21swapper/111:14:361
30909992020,0cyclictest0-21swapper/110:54:351
30909992020,0cyclictest0-21swapper/109:20:011
30909992020,0cyclictest0-21swapper/108:09:361
30909992019,1cyclictest6281-21apt-get07:29:341
30909992019,1cyclictest28660-21apt-get11:34:351
30909992019,1cyclictest23990-21apt-get08:04:351
30909992018,1cyclictest313-21systemd-journal11:05:011
30909992018,1cyclictest313-21systemd-journal11:05:011
30909992018,1cyclictest18405-21munin-run11:10:011
3090999200,0cyclictest0-21swapper/107:44:351
30904992020,0cyclictest0-21swapper/007:34:340
30904992019,1cyclictest0-21swapper/011:39:360
30904992019,1cyclictest0-21swapper/011:09:350
30904992016,4cyclictest0-21swapper/010:37:530
3090499200,20cyclictest0-21swapper/012:24:360
3090499200,0cyclictest0-21swapper/012:39:490
3090499200,0cyclictest0-21swapper/009:59:370
30917991919,0cyclictest31238-21apt-get08:19:363
30917991919,0cyclictest21400-21apt-get11:19:373
30917991919,0cyclictest0-21swapper/312:29:363
30917991919,0cyclictest0-21swapper/311:09:353
30917991919,0cyclictest0-21swapper/310:24:373
30917991919,0cyclictest0-21swapper/309:54:363
30917991919,0cyclictest0-21swapper/309:44:353
30917991919,0cyclictest0-21swapper/308:09:363
30917991919,0cyclictest0-21swapper/308:04:353
30914991919,0cyclictest22669-21apt-get12:29:362
30914991919,0cyclictest0-21swapper/212:09:372
30914991919,0cyclictest0-21swapper/209:44:352
30914991919,0cyclictest0-21swapper/208:39:372
30914991919,0cyclictest0-21swapper/208:19:362
3091499190,18cyclictest18032-21apt-get08:59:342
3091499190,18cyclictest0-21swapper/212:25:002
30909991919,0cyclictest0-21swapper/112:24:361
30909991919,0cyclictest0-21swapper/108:14:361
30909991919,0cyclictest0-21swapper/107:24:361
30909991917,1cyclictest16289-21perf10:00:001
30904991919,0cyclictest0-21swapper/012:34:370
30904991919,0cyclictest0-21swapper/009:44:350
30904991919,0cyclictest0-21swapper/007:24:360
30904991917,1cyclictest12645-21munin-run08:45:010
3090499190,18cyclictest0-21swapper/011:59:340
30917991818,0cyclictest0-21swapper/308:54:363
30917991818,0cyclictest0-21swapper/307:19:363
3091799180,17cyclictest0-21swapper/307:14:353
30914991818,0cyclictest0-21swapper/211:44:362
30914991818,0cyclictest0-21swapper/209:54:362
30914991818,0cyclictest0-21swapper/208:54:362
30914991818,0cyclictest0-21swapper/207:39:362
30914991817,0cyclictest5865-21apt-get11:54:352
30914991816,1cyclictest13995-21apt-get07:44:362
30909991818,0cyclictest0-21swapper/110:39:371
30909991818,0cyclictest0-21swapper/110:34:361
30909991818,0cyclictest0-21swapper/108:29:361
30909991818,0cyclictest0-21swapper/107:19:361
30909991816,1cyclictest829-21cron07:15:011
30909991815,2cyclictest31147-21idleruntime-cro10:30:011
30904991818,0cyclictest0-21swapper/010:14:360
30904991818,0cyclictest0-21swapper/008:24:350
30904991818,0cyclictest0-21swapper/008:04:350
30904991817,0cyclictest15639-21apt-get08:54:360
30904991816,1cyclictest861-21grep07:15:010
3090499180,3cyclictest0-21swapper/009:24:350
3090499180,17cyclictest28224-21apt08:10:000
3091799178,8cyclictest15463-21apt-get12:14:363
30917991717,0cyclictest3888-21apt-get07:24:363
30917991717,0cyclictest0-21swapper/311:54:353
30917991717,0cyclictest0-21swapper/311:11:483
30917991717,0cyclictest0-21swapper/309:14:353
30917991717,0cyclictest0-21swapper/308:49:373
30917991715,1cyclictest19089-21turbostat10:05:013
30917991713,2cyclictest313-21systemd-journal12:05:013
3091799170,0cyclictest19699-21apt-get10:09:343
30914991717,0cyclictest0-21swapper/210:44:362
30914991717,0cyclictest0-21swapper/208:49:372
30909991717,0cyclictest0-21swapper/111:29:361
30909991717,0cyclictest0-21swapper/110:59:361
30909991717,0cyclictest0-21swapper/110:24:371
30909991717,0cyclictest0-21swapper/109:29:351
30909991717,0cyclictest0-21swapper/109:24:351
30909991717,0cyclictest0-21swapper/108:34:351
30909991716,0cyclictest0-21swapper/112:29:361
3090499172,6cyclictest12525-21diskstats09:54:380
30904991717,0cyclictest0-21swapper/012:19:360
30904991717,0cyclictest0-21swapper/010:44:360
30904991717,0cyclictest0-21swapper/007:55:020
30904991716,1cyclictest27636-21apt-get09:19:350
30904991716,0cyclictest0-21swapper/011:44:360
30904991713,3cyclictest0-21swapper/009:29:350
30917991616,0cyclictest21574-21apt-get07:59:353
30917991616,0cyclictest0-21swapper/307:44:363
30917991614,2cyclictest28833-21apt-get08:14:373
30917991614,1cyclictest23894-21munin-run10:15:013
30917991613,2cyclictest6672-21idleruntime-cro09:40:013
30914991616,0cyclictest0-21swapper/209:14:352
30914991616,0cyclictest0-21swapper/207:24:362
30914991614,1cyclictest622-21gdbus09:10:002
30914991614,1cyclictest21189-21apt-get07:55:022
30914991614,1cyclictest19096-21perf10:05:012
30914991614,1cyclictest0-21swapper/210:15:012
30914991614,1cyclictest0-21swapper/209:40:012
3091499160,1cyclictest622-21gdbus09:50:002
30909991616,0cyclictest0-21swapper/108:54:361
30909991616,0cyclictest0-21swapper/108:44:361
30909991615,1cyclictest2498-21apt-get09:34:351
30909991615,0cyclictest0-21swapper/107:54:341
3090999160,15cyclictest10151-21users08:39:461
30904991616,0cyclictest0-21swapper/011:24:350
30904991616,0cyclictest0-21swapper/010:29:350
30904991616,0cyclictest0-21swapper/010:09:340
30904991616,0cyclictest0-21swapper/009:39:350
30904991616,0cyclictest0-21swapper/009:34:350
30904991616,0cyclictest0-21swapper/009:14:350
30904991616,0cyclictest0-21swapper/007:39:360
30904991615,0cyclictest0-21swapper/011:04:360
30904991615,0cyclictest0-21swapper/011:04:360
30904991614,2cyclictest0-21swapper/010:01:230
30904991614,1cyclictest15037-21perf08:50:010
3090499160,14cyclictest0-21swapper/009:01:210
30917991515,0cyclictest0-21swapper/311:39:363
30917991515,0cyclictest0-21swapper/309:19:353
3091799150,14cyclictest10848-21apt-get08:44:363
30914991515,0cyclictest31063-21apt-get11:39:362
30914991515,0cyclictest0-21swapper/208:14:372
3091499150,14cyclictest16263-21irqstats12:14:402
30909991515,0cyclictest0-21swapper/108:59:351
3090499155,1cyclictest3259-21latency_hist07:20:000
3090499152,6cyclictest793-21lldpd12:25:580
3090499150,4cyclictest0-21swapper/011:34:340
3091799140,14cyclictest0-21swapper/312:30:053
3091799140,14cyclictest0-21swapper/312:16:083
3091799140,14cyclictest0-21swapper/311:20:053
3091799140,14cyclictest0-21swapper/309:00:533
3091499140,14cyclictest0-21swapper/212:30:572
3091499140,14cyclictest0-21swapper/212:00:082
3091499140,14cyclictest0-21swapper/211:20:082
3091499140,14cyclictest0-21swapper/211:10:282
3091499140,14cyclictest0-21swapper/211:00:042
3091499140,14cyclictest0-21swapper/211:00:042
3091499140,14cyclictest0-21swapper/209:55:212
3091499140,14cyclictest0-21swapper/209:15:412
3091499140,14cyclictest0-21swapper/208:40:532
3091499140,0cyclictest0-21swapper/211:30:192
3091499140,0cyclictest0-21swapper/210:05:362
30909991414,0cyclictest0-21swapper/111:40:441
30909991414,0cyclictest0-21swapper/109:00:011
3090999140,0cyclictest0-21swapper/109:05:321
3090499140,0cyclictest0-21swapper/009:05:040
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional