You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 23:02

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sun Dec 14, 2025 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28857992622,3cyclictest18567-21apt-get12:24:212
2885899240,23cyclictest11357-21apt-get12:09:213
28857992422,1cyclictest11107-21idleruntime-cro11:00:012
2885799240,2cyclictest31515-21idleruntime-cro10:35:002
2885799240,23cyclictest4505-21apt-get10:49:212
28856992421,2cyclictest23179-21awk12:30:011
2885599240,23cyclictest0-21swapper/007:45:020
2885899230,22cyclictest29865-21turbostat.cron09:25:013
2885899230,22cyclictest20951-21awk08:00:003
2885899230,22cyclictest10585-21awk08:45:003
28857992321,1cyclictest606-21dbus-daemon09:00:012
28857992321,1cyclictest19028-21cron10:10:012
28857992319,3cyclictest8797-21apt-get08:44:212
2885799230,22cyclictest4226-21apt-get07:29:192
2885799230,22cyclictest10971-21runrttasks07:40:012
28856992321,1cyclictest30774-21apt-get11:40:011
2885699231,21cyclictest0-21swapper/107:49:191
2885699230,22cyclictest15530-21cron12:15:011
2885699230,22cyclictest0-21swapper/111:35:011
28855992323,0cyclictest0-21swapper/010:09:210
28855992321,1cyclictest3986-21apt-get08:34:220
28855992321,1cyclictest0-21swapper/007:15:000
28855992320,2cyclictest606-21dbus-daemon09:20:010
28855992320,2cyclictest15556-21grep12:15:010
2885599230,1cyclictest31016-21perf08:20:010
2885899220,21cyclictest8705-21turbostat10:55:013
2885899220,21cyclictest21543-21apt-get08:04:213
2885899220,1cyclictest30454-21apt-get09:29:223
2885899220,1cyclictest20787-21apt-get09:09:213
28857992221,0cyclictest31601-21apt-get08:24:212
28857992220,1cyclictest606-21dbus-daemon12:30:012
28857992220,1cyclictest20691-21tr11:20:002
28857992220,1cyclictest20181-21awk09:05:012
28857992220,1cyclictest10761-21turbostat.cron12:05:002
2885799220,21cyclictest5801-21munin-run08:35:012
2885799220,21cyclictest2113-21apt-get10:44:212
2885799220,21cyclictest1757-21apt-get11:49:232
2885799220,21cyclictest11543-21apt-get07:44:192
2885799220,1cyclictest0-21swapper/211:55:002
28856992220,1cyclictest25582-21apt-get09:19:221
2885699220,21cyclictest6549-21apt-get11:59:221
2885699220,21cyclictest0-21swapper/112:39:211
2885699220,21cyclictest0-21swapper/111:40:021
2885699220,21cyclictest0-21swapper/110:25:001
2885699220,21cyclictest0-21swapper/107:39:201
28855992221,0cyclictest0-21swapper/007:19:200
28855992220,1cyclictest26329-21apt-get08:14:200
2885599220,21cyclictest0-21swapper/007:24:200
2885599220,1cyclictest31772-21apt-get11:44:210
2885599220,1cyclictest17985-21munin-run12:20:010
28858992121,0cyclictest0-21swapper/311:24:223
28858992121,0cyclictest0-21swapper/308:15:023
28858992120,1cyclictest6627-21apt-get07:34:203
28858992120,1cyclictest388-21apt-get09:34:213
28858992119,1cyclictest18315-21perf11:15:003
28858992118,2cyclictest17765-21sendmail09:00:013
2885899210,20cyclictest18397-21apt-get09:04:213
2885899210,1cyclictest17964-21sendmail-msp12:20:013
28857992120,1cyclictest28957-21apt-get11:39:202
28857992120,1cyclictest24852-21apt-get10:24:192
28857992120,0cyclictest0-21swapper/212:19:222
28857992120,0cyclictest0-21swapper/207:34:202
28857992119,2cyclictest0-21swapper/212:35:032
2885799210,20cyclictest11862-21perf09:55:012
2885799210,20cyclictest0-21swapper/211:25:032
2885799210,20cyclictest0-21swapper/208:49:212
2885799210,20cyclictest0-21swapper/208:39:212
2885799210,20cyclictest0-21swapper/208:09:212
2885799210,1cyclictest9447-21turbostat09:50:012
2885799210,1cyclictest29430-21apt-get07:14:212
28856992121,0cyclictest0-21swapper/110:54:211
28856992120,1cyclictest0-21swapper/109:54:201
2885699210,21cyclictest0-21swapper/109:59:211
2885699210,21cyclictest0-21swapper/109:14:201
2885699210,20cyclictest0-21swapper/107:44:191
2885699210,0cyclictest0-21swapper/112:09:201
2885699210,0cyclictest0-21swapper/109:04:211
2885699210,0cyclictest0-21swapper/108:59:211
2885699210,0cyclictest0-21swapper/107:19:201
28855992121,0cyclictest0-21swapper/012:09:210
28855992120,1cyclictest26598-21apt-get12:39:210
28855992120,1cyclictest11712-21apt-get11:04:220
28855992120,1cyclictest0-21swapper/011:54:220
28855992120,1cyclictest0-21swapper/008:24:210
28855992120,0cyclictest0-21swapper/012:04:210
28855992120,0cyclictest0-21swapper/010:54:210
28855992119,1cyclictest0-21swapper/008:39:200
2885599210,20cyclictest0-21swapper/010:49:210
2885599210,20cyclictest0-21swapper/007:59:210
2885599210,0cyclictest0-21swapper/009:49:210
2885599210,0cyclictest0-21swapper/009:29:220
2885599210,0cyclictest0-21swapper/009:24:210
2885599210,0cyclictest0-21swapper/009:04:210
28858992020,0cyclictest0-21swapper/311:39:213
28858992020,0cyclictest0-21swapper/311:04:223
28858992020,0cyclictest0-21swapper/309:49:213
28858992020,0cyclictest0-21swapper/309:19:213
28858992020,0cyclictest0-21swapper/307:29:193
28858992020,0cyclictest0-21swapper/307:24:213
28858992019,1cyclictest5257-21apt-get09:44:213
2885899200,20cyclictest32101-21apt-get10:39:223
2885899200,20cyclictest0-21swapper/311:49:233
2885899200,0cyclictest0-21swapper/310:59:223
2885899200,0cyclictest0-21swapper/310:24:193
2885899200,0cyclictest0-21swapper/308:14:203
28857992020,0cyclictest16499-21apt-get11:14:202
28857992020,0cyclictest13587-21apt-get08:54:212
28857992020,0cyclictest0-21swapper/207:24:212
28856992020,0cyclictest0-21swapper/112:04:211
28856992020,0cyclictest0-21swapper/110:44:211
28856992020,0cyclictest0-21swapper/110:04:221
28856992020,0cyclictest0-21swapper/109:09:211
28856992020,0cyclictest0-21swapper/108:54:221
28856992020,0cyclictest0-21swapper/108:44:211
2885699200,20cyclictest0-21swapper/110:15:031
2885699200,20cyclictest0-21swapper/108:40:001
2885699200,0cyclictest18896-21apt-get11:19:201
2885699200,0cyclictest0-21swapper/111:25:031
28855992020,0cyclictest0-21swapper/011:59:220
28855992020,0cyclictest0-21swapper/009:14:210
28855992020,0cyclictest0-21swapper/008:59:210
28855992020,0cyclictest0-21swapper/008:04:210
28855992019,1cyclictest23758-21apt-get12:34:240
2885599200,20cyclictest0-21swapper/010:39:220
28858991919,0cyclictest4156-21apt-get11:54:223
28858991919,0cyclictest1575-21apt-get08:29:213
28858991919,0cyclictest0-21swapper/311:29:213
28858991917,1cyclictest25554-21apt-get12:35:013
28858991917,1cyclictest10785-21munin-run12:05:003
28858991917,1cyclictest0-21swapper/310:05:013
28857991919,0cyclictest28052-21apt-get09:24:202
28857991919,0cyclictest27304-21apt-get10:29:232
28857991919,0cyclictest0-21swapper/209:44:222
28857991919,0cyclictest0-21swapper/208:14:202
28857991918,1cyclictest14106-21apt-get11:09:212
28857991918,1cyclictest0-21swapper/209:59:212
28857991917,1cyclictest0-21swapper/210:54:212
2885799190,0cyclictest0-21swapper/211:24:222
2885799190,0cyclictest0-21swapper/210:39:212
28856991919,0cyclictest0-21swapper/110:34:221
28856991919,0cyclictest0-21swapper/109:38:021
28856991919,0cyclictest0-21swapper/108:34:221
28856991919,0cyclictest0-21swapper/108:19:211
28856991919,0cyclictest0-21swapper/107:54:211
28856991917,1cyclictest23131-21munin-run11:25:001
28855991919,0cyclictest0-21swapper/011:49:230
28855991919,0cyclictest0-21swapper/010:24:190
28855991919,0cyclictest0-21swapper/008:54:210
28855991919,0cyclictest0-21swapper/008:49:210
28855991919,0cyclictest0-21swapper/008:09:220
28858991818,0cyclictest22437-21apt-get10:19:213
28858991818,0cyclictest0-21swapper/311:59:223
28858991818,0cyclictest0-21swapper/311:44:213
28858991818,0cyclictest0-21swapper/311:09:223
28858991818,0cyclictest0-21swapper/310:44:213
28858991818,0cyclictest0-21swapper/309:14:213
28858991818,0cyclictest0-21swapper/308:39:213
28858991818,0cyclictest0-21swapper/308:24:203
28858991817,1cyclictest21357-21apt-get12:29:233
28858991817,1cyclictest13353-21apt-get07:45:023
28858991816,1cyclictest6324-21munin-run10:50:013
28857991818,0cyclictest0-21swapper/209:34:212
28857991818,0cyclictest0-21swapper/209:19:212
2885799180,0cyclictest26552-21apt-get11:34:202
28856991818,0cyclictest0-21swapper/111:14:201
28856991818,0cyclictest0-21swapper/110:29:231
28856991818,0cyclictest0-21swapper/110:14:211
28856991818,0cyclictest0-21swapper/109:34:211
28856991818,0cyclictest0-21swapper/108:29:211
28856991816,1cyclictest9458-21perf09:50:011
28855991818,0cyclictest0-21swapper/008:44:210
28855991817,1cyclictest0-21swapper/012:29:230
28855991816,1cyclictest606-21dbus-daemon12:25:010
28855991816,1cyclictest15896-21sh11:10:000
28858991717,0cyclictest16732-21apt-get07:54:213
28858991717,0cyclictest0-21swapper/308:49:213
28858991715,1cyclictest28376-21perf11:35:013
28857991717,0cyclictest23758-21apt-get12:34:242
28857991717,0cyclictest0-21swapper/210:04:222
28857991717,0cyclictest0-21swapper/209:39:212
2885799170,0cyclictest0-21swapper/211:04:222
2885799170,0cyclictest0-21swapper/209:29:222
28856991717,0cyclictest0-21swapper/110:39:221
28856991717,0cyclictest0-21swapper/107:29:191
28856991717,0cyclictest0-21swapper/107:24:201
28856991715,1cyclictest11190-21apt-get08:49:211
28855991717,0cyclictest0-21swapper/011:39:210
28855991717,0cyclictest0-21swapper/007:29:190
28855991716,1cyclictest3922-21perf10:45:010
28855991715,1cyclictest8442-21munin-plugin-st07:35:010
2885599170,16cyclictest16832-21cat10:05:010
28858991616,0cyclictest9128-21apt-get07:39:203
28858991616,0cyclictest0-21swapper/312:35:033
28858991616,0cyclictest0-21swapper/312:24:223
28858991616,0cyclictest0-21swapper/312:15:013
28858991616,0cyclictest0-21swapper/310:34:223
28858991616,0cyclictest0-21swapper/309:59:213
28858991616,0cyclictest0-21swapper/308:54:213
28858991616,0cyclictest0-21swapper/307:18:393
28858991615,1cyclictest18896-21apt-get11:19:213
28858991614,1cyclictest25746-21latency_hist08:10:013
28857991616,0cyclictest0-21swapper/208:19:212
28857991616,0cyclictest0-21swapper/208:04:222
28857991615,0cyclictest0-21swapper/211:59:222
28857991614,2cyclictest0-21swapper/210:17:502
2885799160,15cyclictest3306-21users08:29:312
28856991616,0cyclictest0-21swapper/112:24:211
28856991616,0cyclictest0-21swapper/111:09:211
28856991614,1cyclictest0-21swapper/108:04:301
2885699160,15cyclictest9296-21apt-get10:59:221
28855991616,0cyclictest0-21swapper/011:24:220
28855991616,0cyclictest0-21swapper/009:44:210
28855991616,0cyclictest0-21swapper/009:09:200
28855991614,1cyclictest0-21swapper/011:14:290
28855991614,1cyclictest0-21swapper/010:29:330
2885599160,2cyclictest21176-21cat10:14:300
2885599160,15cyclictest0-21swapper/007:39:220
28858991515,0cyclictest0-21swapper/310:29:233
28858991515,0cyclictest0-21swapper/310:14:213
28858991515,0cyclictest0-21swapper/307:44:193
28858991513,1cyclictest18296-21ntp_states10:09:273
28857991515,0cyclictest0-21swapper/212:09:212
28857991515,0cyclictest0-21swapper/211:44:212
28857991514,1cyclictest0-21swapper/207:16:252
28857991514,0cyclictest0-21swapper/207:45:022
2885799150,1cyclictest25005-21munin-run09:15:012
28856991515,0cyclictest0-21swapper/112:34:251
28856991515,0cyclictest0-21swapper/111:04:221
28856991515,0cyclictest0-21swapper/107:59:201
28856991513,1cyclictest7076-21munin-run09:45:001
28856991512,2cyclictest31225-21cron07:15:001
28856991512,2cyclictest30650-21cpuspeed_turbos09:29:221
2885699150,14cyclictest0-21swapper/110:49:301
28855991515,0cyclictest0-21swapper/010:34:220
28855991515,0cyclictest0-21swapper/010:19:210
28855991515,0cyclictest0-21swapper/009:54:200
28855991515,0cyclictest0-21swapper/008:29:470
28855991513,1cyclictest0-21swapper/011:19:270
2885899140,14cyclictest0-21swapper/309:50:353
2885899140,14cyclictest0-21swapper/309:36:323
2885899140,14cyclictest0-21swapper/308:31:473
28857991413,1cyclictest0-21swapper/209:05:332
2885799140,14cyclictest0-21swapper/212:10:122
2885799140,14cyclictest0-21swapper/207:56:282
28856991414,0cyclictest0-21swapper/110:05:111
28856991414,0cyclictest0-21swapper/109:20:311
28856991414,0cyclictest0-21swapper/108:10:141
2885699140,14cyclictest0-21swapper/111:50:271
2885699140,0cyclictest0-21swapper/112:15:461
2885699140,0cyclictest0-21swapper/107:30:491
28855991414,0cyclictest0-21swapper/010:56:210
28855991413,1cyclictest23113-21apt-get11:25:030
2885599140,14cyclictest0-21swapper/011:30:340
2885599140,14cyclictest0-21swapper/009:55:030
2885599140,14cyclictest0-21swapper/009:30:070
2885599140,14cyclictest0-21swapper/007:50:210
2885599140,0cyclictest0-21swapper/009:35:400
2885599140,0cyclictest0-21swapper/007:40:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional