You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-15 - 17:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Mon Dec 15, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2842499260,25cyclictest18893-21apt-get11:19:260
2842899250,24cyclictest0-21swapper/112:34:261
2843499240,23cyclictest24951-21dump-pmu-power08:10:013
2843099241,22cyclictest606-21dbus-daemon09:00:012
2842499240,23cyclictest17245-21apt10:09:260
28434992321,1cyclictest1647-21apt-get10:44:263
28434992320,2cyclictest3188-21dump-pmu-power07:25:013
2843099230,23cyclictest0-21swapper/207:14:242
2843099230,22cyclictest29760-21munin-run08:20:002
2843099230,22cyclictest2694-21basename11:50:002
28428992323,0cyclictest0-21swapper/111:19:261
28428992321,1cyclictest22540-21latency_hist08:05:011
2842899230,22cyclictest27191-21cron12:40:001
28424992322,1cyclictest0-21swapper/011:49:270
28424992321,1cyclictest5863-21munin-run10:50:000
28424992320,2cyclictest15435-21awk11:10:010
28424992320,1cyclictest313-21systemd-journal10:25:010
2842499230,22cyclictest7907-21latency_hist12:00:010
2842499230,22cyclictest29808-21perf09:25:010
28434992220,1cyclictest606-21dbus-daemon11:15:013
28434992220,1cyclictest0-21swapper/311:25:013
2843499220,21cyclictest13647-21apt-get11:09:243
2843499220,21cyclictest0-21swapper/311:55:013
2843499220,1cyclictest12383-21apt-get09:59:263
2843499220,1cyclictest0-21swapper/310:05:003
2843499220,1cyclictest0-21swapper/309:10:003
28430992222,0cyclictest0-21swapper/207:19:272
28430992220,1cyclictest2141-21perf08:30:002
2843099220,21cyclictest6446-21apt-get10:54:262
2843099220,21cyclictest1005-21sendmail10:40:012
28428992220,1cyclictest15449-21apt-get11:10:021
28428992218,3cyclictest15444-21sh11:10:011
2842899220,21cyclictest27999-21apt-get09:24:251
2842899220,21cyclictest0-21swapper/110:54:261
2842899220,1cyclictest6954-21apt-get08:40:021
2842899220,1cyclictest32756-21apt-get08:29:261
28424992221,1cyclictest0-21swapper/012:04:250
28424992220,1cyclictest12153-21munin-plugin-st08:50:010
28424992220,1cyclictest0-21swapper/010:29:260
2842499220,21cyclictest7948-21apt-get08:44:270
2842499220,21cyclictest3779-21apt-get07:29:230
2842499220,21cyclictest30360-21apt-get08:24:260
2842499220,21cyclictest10672-21awk11:00:000
2842499220,21cyclictest0-21swapper/012:24:270
2842499220,21cyclictest0-21swapper/010:34:280
2842499220,21cyclictest0-21swapper/009:19:260
2842499220,21cyclictest0-21swapper/008:29:270
2842499220,1cyclictest0-21swapper/011:50:020
2842499220,1cyclictest0-21swapper/011:29:270
2842499220,1cyclictest0-21swapper/009:49:250
2842499220,1cyclictest0-21swapper/009:09:250
28434992121,0cyclictest28985-21apt-get07:14:253
28434992119,1cyclictest606-21dbus-daemon12:05:013
28434992119,1cyclictest27906-21munin-run11:35:003
28434992118,2cyclictest15090-21cat12:15:013
2843499210,20cyclictest0-21swapper/311:49:273
2843499210,20cyclictest0-21swapper/307:29:233
2843499210,1cyclictest0-21swapper/308:24:263
28430992121,0cyclictest18212-21apt-get12:24:272
28430992121,0cyclictest0-21swapper/210:04:252
28430992121,0cyclictest0-21swapper/207:54:242
28430992120,1cyclictest6954-21apt-get08:40:012
28430992120,1cyclictest20703-21sendmail-msp11:20:012
28430992120,1cyclictest0-21swapper/211:34:262
28430992120,0cyclictest25582-21apt-get09:19:262
28430992120,0cyclictest0-21swapper/211:09:242
2843099210,20cyclictest18333-21apt-get07:59:252
2843099210,0cyclictest0-21swapper/210:14:272
28428992121,0cyclictest0-21swapper/111:50:021
28428992121,0cyclictest0-21swapper/110:19:261
28428992121,0cyclictest0-21swapper/109:49:251
28428992121,0cyclictest0-21swapper/109:44:251
28428992121,0cyclictest0-21swapper/109:29:251
28428992120,0cyclictest0-21swapper/108:24:261
28428992119,1cyclictest894-21apt-get11:49:271
28428992119,1cyclictest10670-21perf11:00:001
2842899210,21cyclictest0-21swapper/107:29:231
2842899210,20cyclictest17612-21sendmail-msp12:20:011
2842899210,0cyclictest4050-21apt-get10:49:251
2842899210,0cyclictest1375-21apt-get07:24:251
2842899210,0cyclictest0-21swapper/111:39:271
2842899210,0cyclictest0-21swapper/110:44:261
2842899210,0cyclictest0-21swapper/108:19:251
28424992121,0cyclictest0-21swapper/011:24:260
28424992120,0cyclictest0-21swapper/007:49:270
28424992119,1cyclictest606-21dbus-daemon09:30:010
28424992119,1cyclictest20152-21basename08:00:010
2842499210,1cyclictest1-21systemd08:05:010
2842499210,1cyclictest0-21swapper/009:34:240
28434992020,0cyclictest0-21swapper/309:34:243
28434992020,0cyclictest0-21swapper/309:24:253
28434992020,0cyclictest0-21swapper/308:55:023
28434992019,1cyclictest25509-21awk11:30:013
28434992019,1cyclictest25397-21apt-get12:39:273
28434992019,1cyclictest25205-21apt-get09:15:023
28434992019,1cyclictest20605-21apt-get12:29:263
28434992019,1cyclictest17246-21apt-get10:09:263
2843499200,0cyclictest6178-21apt-get07:34:253
2843499200,0cyclictest5200-21apt-get09:44:253
2843499200,0cyclictest0-21swapper/311:44:263
2843499200,0cyclictest0-21swapper/310:29:263
2843499200,0cyclictest0-21swapper/307:44:263
28430992020,0cyclictest0-21swapper/211:54:272
28430992020,0cyclictest0-21swapper/210:49:252
28430992020,0cyclictest0-21swapper/209:29:252
28430992020,0cyclictest0-21swapper/209:24:242
28430992020,0cyclictest0-21swapper/208:44:272
28430992019,1cyclictest10901-21apt-get12:09:262
28430992019,1cyclictest0-21swapper/209:44:252
28430992019,0cyclictest20391-21apt-get09:09:252
28430992018,1cyclictest14172-21cron10:00:002
2843099200,0cyclictest8506-21apt-get12:04:252
2843099200,0cyclictest2784-21apt-get09:39:262
2843099200,0cyclictest0-21swapper/211:44:262
2843099200,0cyclictest0-21swapper/211:39:272
2843099200,0cyclictest0-21swapper/211:04:272
28428992020,0cyclictest26848-21apt-get10:29:261
28428992020,0cyclictest0-21swapper/111:24:261
28428992020,0cyclictest0-21swapper/110:24:271
28428992020,0cyclictest0-21swapper/109:34:241
28428992020,0cyclictest0-21swapper/108:34:251
28428992020,0cyclictest0-21swapper/107:49:271
28428992020,0cyclictest0-21swapper/107:39:251
28428992020,0cyclictest0-21swapper/107:34:251
28428992019,0cyclictest29255-21apt-get10:34:281
2842899200,20cyclictest0-21swapper/109:09:251
2842899200,0cyclictest0-21swapper/112:09:261
2842899200,0cyclictest0-21swapper/111:29:271
28424992020,0cyclictest0-21swapper/012:18:490
28424992020,0cyclictest0-21swapper/010:19:260
28424992020,0cyclictest0-21swapper/009:59:260
28424992020,0cyclictest0-21swapper/009:04:250
28424992020,0cyclictest0-21swapper/008:34:240
28424992019,1cyclictest0-21swapper/007:39:240
28424992019,0cyclictest0-21swapper/008:59:250
28424992018,1cyclictest16646-21turbostat10:05:000
2842499200,19cyclictest12739-21apt-get08:54:250
2842499200,0cyclictest0-21swapper/011:10:020
28434991919,0cyclictest31385-21apt-get07:19:273
28434991919,0cyclictest2743-21apt-get08:34:253
28434991919,0cyclictest0-21swapper/310:59:273
28434991919,0cyclictest0-21swapper/310:34:283
28434991919,0cyclictest0-21swapper/308:54:253
28434991919,0cyclictest0-21swapper/308:39:263
28434991918,0cyclictest0-21swapper/309:54:263
28430991919,0cyclictest0-21swapper/211:14:252
28430991919,0cyclictest0-21swapper/210:09:262
28430991919,0cyclictest0-21swapper/209:15:002
28430991918,1cyclictest0-21swapper/208:24:262
28430991917,1cyclictest3467-21munin-run10:45:002
2843099190,18cyclictest0-21swapper/212:29:262
28428991919,0cyclictest0-21swapper/112:24:271
28428991919,0cyclictest0-21swapper/109:19:261
28428991919,0cyclictest0-21swapper/108:39:251
28428991918,1cyclictest22-21ksoftirqd/110:05:001
28428991917,1cyclictest12119-21cron08:50:011
28424991919,0cyclictest0-21swapper/012:39:270
28424991919,0cyclictest0-21swapper/012:34:260
28424991919,0cyclictest0-21swapper/010:39:270
28424991919,0cyclictest0-21swapper/010:14:270
28424991919,0cyclictest0-21swapper/007:54:240
28424991918,0cyclictest11094-21apt-get07:44:260
28424991916,2cyclictest6972-21awk08:40:010
28434991818,0cyclictest0-21swapper/312:24:273
28434991818,0cyclictest0-21swapper/307:54:243
2843499180,0cyclictest17999-21apt-get09:04:253
28430991818,0cyclictest0-21swapper/212:39:272
28430991818,0cyclictest0-21swapper/211:24:262
28430991818,0cyclictest0-21swapper/207:29:232
28430991817,1cyclictest0-21swapper/207:50:012
2842899182,15cyclictest22-21ksoftirqd/107:45:001
28428991818,0cyclictest0-21swapper/112:04:251
28428991818,0cyclictest0-21swapper/109:59:261
28428991817,1cyclictest0-21swapper/109:04:251
28428991817,0cyclictest0-21swapper/108:54:251
28428991816,1cyclictest20156-21latency_hist08:00:011
28424991818,0cyclictest0-21swapper/012:09:260
28424991818,0cyclictest0-21swapper/009:54:260
28424991818,0cyclictest0-21swapper/007:14:240
28424991817,1cyclictest0-21swapper/011:34:260
28424991815,2cyclictest4569-21cron09:40:010
28434991717,0cyclictest0-21swapper/312:19:273
28434991717,0cyclictest0-21swapper/311:19:273
28434991717,0cyclictest0-21swapper/307:39:253
28434991714,2cyclictest8252-21latency_hist10:55:003
2843499170,16cyclictest0-21swapper/308:19:253
2843499170,0cyclictest10343-21apt-get08:49:263
28430991717,0cyclictest0-21swapper/210:59:272
28430991717,0cyclictest0-21swapper/208:09:252
28430991717,0cyclictest0-21swapper/208:04:252
28430991715,1cyclictest330-21apt-get09:34:242
28428991717,0cyclictest0-21swapper/107:54:241
28424991717,0cyclictest0-21swapper/009:44:250
28424991717,0cyclictest0-21swapper/008:19:250
28424991717,0cyclictest0-21swapper/007:34:240
28424991715,1cyclictest606-21dbus-daemon09:15:000
2842499170,0cyclictest0-21swapper/007:24:250
28434991616,0cyclictest23004-21apt-get12:34:263
28434991616,0cyclictest0-21swapper/311:39:273
28434991616,0cyclictest0-21swapper/310:49:253
28434991615,1cyclictest20748-21apt-get08:04:253
28434991614,1cyclictest26242-21dump-pmu-power10:25:013
28430991616,0cyclictest0-21swapper/209:04:252
28430991616,0cyclictest0-21swapper/208:14:252
28430991615,1cyclictest24791-21sh12:35:002
28430991614,1cyclictest28624-21cron10:30:012
2843099160,15cyclictest13044-21runrttasks12:11:352
28428991616,0cyclictest0-21swapper/110:39:271
28428991616,0cyclictest0-21swapper/108:59:251
28428991614,1cyclictest27955-21munin-run11:35:001
28424991616,0cyclictest0-21swapper/008:14:240
28424991616,0cyclictest0-21swapper/008:09:250
28424991614,1cyclictest606-21dbus-daemon11:05:010
2842499160,15cyclictest0-21swapper/010:51:020
28434991515,0cyclictest19641-21apt-get10:14:273
28434991515,0cyclictest0-21swapper/308:00:013
2843499150,14cyclictest12302-21ntp_states11:04:323
28430991515,0cyclictest0-21swapper/212:19:262
28430991515,0cyclictest0-21swapper/210:19:262
28430991515,0cyclictest0-21swapper/208:54:252
28430991515,0cyclictest0-21swapper/208:34:252
28430991512,3cyclictest0-21swapper/208:45:032
28428991515,0cyclictest0-21swapper/109:54:261
28428991515,0cyclictest0-21swapper/107:14:241
2842899150,14cyclictest0-21swapper/111:01:461
28434991414,0cyclictest0-21swapper/309:10:473
2843499140,14cyclictest0-21swapper/311:55:013
2843499140,14cyclictest0-21swapper/310:35:133
2843499140,14cyclictest0-21swapper/310:15:283
2843499140,14cyclictest0-21swapper/309:36:123
2843499140,14cyclictest0-21swapper/309:25:183
2843499140,14cyclictest0-21swapper/308:25:183
2843499140,14cyclictest0-21swapper/308:10:263
2843499140,14cyclictest0-21swapper/307:45:383
2843499140,0cyclictest0-21swapper/312:05:253
2843499140,0cyclictest0-21swapper/308:41:143
28430991414,0cyclictest0-21swapper/210:30:532
28430991414,0cyclictest0-21swapper/209:50:042
28430991414,0cyclictest0-21swapper/209:45:052
28430991414,0cyclictest0-21swapper/207:40:212
28430991414,0cyclictest0-21swapper/207:35:222
28430991414,0cyclictest0-21swapper/207:20:232
2843099140,0cyclictest0-21swapper/211:55:022
2843099140,0cyclictest0-21swapper/211:27:222
2843099140,0cyclictest0-21swapper/210:20:132
2843099140,0cyclictest0-21swapper/207:32:422
28428991414,0cyclictest0-21swapper/108:10:151
2842899140,14cyclictest0-21swapper/112:25:141
2842899140,14cyclictest0-21swapper/112:10:051
2842899140,14cyclictest0-21swapper/111:40:531
2842899140,14cyclictest0-21swapper/110:06:561
2842899140,0cyclictest0-21swapper/108:06:131
28424991414,0cyclictest0-21swapper/012:10:290
28424991414,0cyclictest0-21swapper/011:35:020
28424991414,0cyclictest0-21swapper/007:15:440
2842499140,14cyclictest0-21swapper/011:40:020
2842499140,14cyclictest0-21swapper/010:40:080
2842499140,0cyclictest0-21swapper/012:25:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional