You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-07 - 23:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Wed Jan 07, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27569992523,1cyclictest28-21ksoftirqd/210:16:252
27568992522,2cyclictest22-21ksoftirqd/108:20:011
27568992424,0cyclictest0-21swapper/107:26:251
27568992422,1cyclictest17719-21perf09:00:011
2756899240,23cyclictest19403-21perf11:15:001
2756499240,2cyclictest1763-21perf10:40:010
2756499240,24cyclictest0-21swapper/007:11:230
27570992320,2cyclictest313-21systemd-journal10:00:013
2757099230,22cyclictest21692-21basename08:05:003
27569992323,0cyclictest0-21swapper/212:36:252
27569992321,1cyclictest13209-21apt-get08:51:242
27569992320,0cyclictest151rcu_preempt07:36:242
2756999230,22cyclictest19356-21latency_hist10:10:012
2756999230,22cyclictest19255-21sh08:00:002
2756999230,1cyclictest32691-21apt-get08:25:022
27568992320,2cyclictest606-21dbus-daemon08:35:011
27568992320,2cyclictest27268-21cron09:20:011
2756899230,22cyclictest12076-21cron09:55:011
2756499231,1cyclictest607-21in:imuxsock07:35:010
27570992222,0cyclictest0-21swapper/309:16:233
27570992221,0cyclictest0-21swapper/311:50:013
27570992220,1cyclictest313-21systemd-journal11:30:013
27570992220,1cyclictest24911-21latency_hist09:15:013
27570992220,1cyclictest11754-21awk11:00:013
2757099220,21cyclictest9612-21latency_hist07:40:013
2757099220,21cyclictest26960-21apt-get11:31:243
2757099220,21cyclictest24530-21apt-get10:21:243
2757099220,21cyclictest20662-21apt-get12:25:023
2757099220,21cyclictest0-21swapper/310:36:243
2757099220,1cyclictest29364-21apt-get11:36:243
2757099220,1cyclictest2454-21grep08:30:013
2757099220,1cyclictest18223-21cron12:20:013
2757099220,1cyclictest0-21swapper/308:41:243
2756999220,21cyclictest13818-21apt-get12:11:242
2756999220,21cyclictest0-21swapper/211:56:232
2756999220,21cyclictest0-21swapper/209:50:012
2756999220,21cyclictest0-21swapper/209:26:242
27568992220,1cyclictest10058-21wc08:45:001
2756899220,21cyclictest4555-21perf10:45:011
2756899220,21cyclictest25824-21cron12:35:011
2756899220,21cyclictest24828-21apt-get08:11:241
2756899220,21cyclictest0-21swapper/108:50:021
2756899220,1cyclictest9632-21munin-run07:40:011
2756899220,1cyclictest22-21ksoftirqd/111:01:231
27564992221,0cyclictest0-21swapper/007:26:250
27564992220,1cyclictest29277-21apt-get08:20:020
27564992220,1cyclictest12460-21apt-get09:56:230
27564992220,1cyclictest0-21swapper/009:45:000
27564992220,1cyclictest0-21swapper/009:20:020
27564992220,1cyclictest0-21swapper/008:20:010
2756499220,21cyclictest1757-21apt-get10:40:020
2756499220,21cyclictest12385-21apt-get07:46:240
2756499220,21cyclictest10997-21cron12:05:000
2756499220,21cyclictest0-21swapper/012:26:260
2756499220,1cyclictest7258-21apt-get08:40:030
2756499220,1cyclictest14203-21turbostat.cron11:05:000
2756499220,1cyclictest0-21swapper/011:16:230
2756499220,1cyclictest0-21swapper/008:05:020
2756499220,0cyclictest25260-21apt-get09:16:230
27570992121,0cyclictest0-21swapper/312:11:243
27570992120,1cyclictest30477-21apt-get09:26:243
27570992120,1cyclictest10418-21apt-get08:46:233
27570992119,1cyclictest6171-21cron11:55:013
27570992119,1cyclictest22168-21apt-get11:21:223
27570992119,1cyclictest0-21swapper/310:41:263
2757099210,20cyclictest0-21swapper/312:31:263
2757099210,20cyclictest0-21swapper/312:06:243
2757099210,20cyclictest0-21swapper/310:26:243
2757099210,20cyclictest0-21swapper/309:51:263
2757099210,0cyclictest17325-21apt-get10:06:253
2757099210,0cyclictest0-21swapper/310:51:243
27569992121,0cyclictest0-21swapper/211:11:242
27569992121,0cyclictest0-21swapper/207:41:232
27569992120,1cyclictest0-21swapper/210:40:022
27569992120,0cyclictest5064-21apt-get07:31:212
27569992120,0cyclictest4162-21apt-get11:51:242
27569992120,0cyclictest0-21swapper/212:21:252
27569992120,0cyclictest0-21swapper/209:56:232
27569992119,1cyclictest14830-21apt-get07:51:222
2756999210,20cyclictest12121-21apt-get11:01:232
2756999210,1cyclictest2814-21apt-get08:31:222
2756999210,0cyclictest0-21swapper/210:31:252
27568992121,0cyclictest5210-21apt-get08:36:231
27568992121,0cyclictest0-21swapper/110:06:241
27568992121,0cyclictest0-21swapper/107:31:211
27568992120,1cyclictest32119-21apt-get10:36:241
27568992120,0cyclictest0-21swapper/111:36:241
27568992120,0cyclictest0-21swapper/107:56:231
27568992119,1cyclictest31336-21cron10:35:001
2756899211,1cyclictest211rcuc/112:16:231
2756899210,20cyclictest30265-21apt-get07:16:251
2756899210,20cyclictest0-21swapper/112:01:241
2756899210,20cyclictest0-21swapper/111:41:251
2756899210,20cyclictest0-21swapper/110:21:241
2756899210,20cyclictest0-21swapper/108:46:231
2756899210,20cyclictest0-21swapper/107:46:241
2756899210,20cyclictest0-21swapper/107:11:231
27564992119,1cyclictest15299-21latency_hist08:55:000
27564992118,2cyclictest15846-21turbostat.cron12:15:000
27564992118,2cyclictest12096-21dump-pmu-power09:55:010
2756499210,20cyclictest26923-21apt-get10:26:240
2756499210,20cyclictest0-21swapper/011:51:240
2756499210,20cyclictest0-21swapper/011:41:250
2756499210,20cyclictest0-21swapper/010:21:240
2756499210,1cyclictest1764-21apt-get11:46:240
2756499210,0cyclictest0-21swapper/011:11:240
2756499210,0cyclictest0-21swapper/011:05:020
2756499210,0cyclictest0-21swapper/007:51:220
27570992020,0cyclictest0-21swapper/310:11:253
27570992020,0cyclictest0-21swapper/308:05:023
27570992020,0cyclictest0-21swapper/307:41:233
27570992019,1cyclictest26213-21apt-get12:36:253
27570992019,1cyclictest0-21swapper/307:11:233
27570992018,1cyclictest17007-21munin-plugin-st11:10:003
2757099200,20cyclictest0-21swapper/308:36:233
2757099200,0cyclictest31782-21apt-get11:41:253
27569992020,0cyclictest4913-21apt-get10:46:242
27569992020,0cyclictest0-21swapper/211:41:252
27569992020,0cyclictest0-21swapper/210:26:242
27569992020,0cyclictest0-21swapper/209:36:242
27569992020,0cyclictest0-21swapper/208:56:252
27569992018,1cyclictest21282-21apt-get12:25:022
27569992018,1cyclictest19647-21apt-get08:01:262
2756999200,0cyclictest0-21swapper/212:16:232
2756999200,0cyclictest0-21swapper/208:46:232
2756999200,0cyclictest0-21swapper/207:16:242
27568992020,0cyclictest0-21swapper/111:06:251
27568992020,0cyclictest0-21swapper/110:46:241
27568992020,0cyclictest0-21swapper/109:50:011
27568992020,0cyclictest0-21swapper/109:21:241
27568992020,0cyclictest0-21swapper/108:21:231
27568992019,1cyclictest22-21ksoftirqd/111:16:221
27568992018,2cyclictest24147-21perf10:20:001
2756899200,20cyclictest0-21swapper/108:01:261
27564992019,0cyclictest0-21swapper/008:11:230
2756499200,0cyclictest0-21swapper/007:56:240
27570991919,0cyclictest19749-21apt-get11:16:233
27570991919,0cyclictest0-21swapper/312:01:243
27570991919,0cyclictest0-21swapper/307:51:223
27570991919,0cyclictest0-21swapper/307:21:243
27570991917,1cyclictest29258-21dump-pmu-power08:20:013
27569991919,0cyclictest27220-21apt-get08:16:232
27569991919,0cyclictest0-21swapper/211:21:222
27569991919,0cyclictest0-21swapper/210:51:242
27569991919,0cyclictest0-21swapper/208:11:242
27569991917,1cyclictest4696-21apt07:30:012
27569991917,1cyclictest22506-21perf09:10:012
27569991917,1cyclictest0-21swapper/211:46:252
27569991917,1cyclictest0-21swapper/208:40:032
27568991919,0cyclictest0-21swapper/110:11:251
27568991919,0cyclictest0-21swapper/109:01:251
27568991919,0cyclictest0-21swapper/108:26:241
27568991919,0cyclictest0-21swapper/107:51:221
27568991918,1cyclictest0-21swapper/112:25:021
27568991917,1cyclictest26213-21apt-get12:36:261
27568991917,1cyclictest24561-21apt-get11:26:221
27568991916,2cyclictest22-21ksoftirqd/112:10:011
27568991915,1cyclictest22-21ksoftirqd/111:31:321
27564991919,0cyclictest0-21swapper/011:21:220
27564991919,0cyclictest0-21swapper/009:26:240
27564991919,0cyclictest0-21swapper/009:11:250
27564991919,0cyclictest0-21swapper/008:46:230
27564991919,0cyclictest0-21swapper/008:31:220
27564991918,1cyclictest16979-21munin-run10:05:010
27564991918,0cyclictest0-21swapper/007:36:240
27570991818,0cyclictest17227-21apt-get07:56:243
27570991818,0cyclictest0-21swapper/311:56:233
27570991818,0cyclictest0-21swapper/308:31:223
27570991818,0cyclictest0-21swapper/307:31:213
27570991818,0cyclictest0-21swapper/307:16:243
27570991817,0cyclictest29328-21apt-get10:31:253
27570991816,1cyclictest2862-21apt-get09:36:233
27570991816,1cyclictest17671-21cron09:00:003
27569991818,0cyclictest19719-21apt-get10:11:242
27569991818,0cyclictest0-21swapper/211:26:222
27569991818,0cyclictest0-21swapper/210:36:242
27569991818,0cyclictest0-21swapper/208:36:232
27569991817,1cyclictest5279-21apt-get09:41:242
27569991817,1cyclictest10068-21apt-get09:51:262
27568991818,0cyclictest0-21swapper/110:56:251
27568991818,0cyclictest0-21swapper/107:41:241
27564991818,0cyclictest0-21swapper/008:56:250
27564991818,0cyclictest0-21swapper/008:25:020
27564991818,0cyclictest0-21swapper/007:41:230
27564991816,1cyclictest2318-21wc07:25:010
27564991812,5cyclictest2195-21vmstat09:31:320
27570991717,0cyclictest0-21swapper/310:01:233
27570991717,0cyclictest0-21swapper/309:21:243
27570991717,0cyclictest0-21swapper/309:06:233
27570991717,0cyclictest0-21swapper/308:50:023
27570991717,0cyclictest0-21swapper/307:26:253
27569991717,0cyclictest12385-21apt-get07:46:242
27569991717,0cyclictest0-21swapper/211:31:252
27569991717,0cyclictest0-21swapper/208:21:232
27569991715,1cyclictest21678-21apt-get08:05:022
27568991717,0cyclictest0-21swapper/112:11:241
27568991717,0cyclictest0-21swapper/109:31:221
27568991717,0cyclictest0-21swapper/109:06:231
27568991716,1cyclictest22-21ksoftirqd/111:50:011
27568991716,0cyclictest0-21swapper/109:56:231
27568991714,2cyclictest24473-21ls08:10:001
27564991717,0cyclictest0-21swapper/011:36:240
27564991717,0cyclictest0-21swapper/010:11:290
27570991616,0cyclictest0-21swapper/309:46:233
27569991616,0cyclictest0-21swapper/211:36:242
27569991616,0cyclictest0-21swapper/209:01:252
27569991616,0cyclictest0-21swapper/207:21:252
2756999160,2cyclictest1783-21mailstats09:31:292
2756999160,15cyclictest0-21swapper/212:05:002
27568991615,1cyclictest0-21swapper/109:41:251
27568991614,1cyclictest2862-21apt-get09:36:231
27564991616,0cyclictest0-21swapper/012:20:010
27564991615,1cyclictest23817-21apt-get12:31:250
27564991615,0cyclictest0-21swapper/011:56:230
27564991615,0cyclictest0-21swapper/011:26:210
27564991614,1cyclictest21677-21perf08:05:000
27570991515,0cyclictest0-21swapper/311:01:233
27570991515,0cyclictest0-21swapper/310:46:243
2757099150,15cyclictest0-21swapper/308:23:143
2757099150,14cyclictest0-21swapper/307:46:273
27569991513,1cyclictest27379-21date09:20:012
2756899155,9cyclictest22-21ksoftirqd/110:30:011
27568991515,0cyclictest0-21swapper/111:51:241
27568991514,1cyclictest14931-21apt-get10:01:231
2756899150,0cyclictest32684-21apt-get07:21:241
27564991515,0cyclictest20119-21perf09:05:010
27564991515,0cyclictest0-21swapper/012:06:240
27564991515,0cyclictest0-21swapper/010:46:240
27564991515,0cyclictest0-21swapper/010:31:250
27564991514,1cyclictest20469-21apt-get09:06:230
27564991514,1cyclictest0-21swapper/012:40:010
27564991513,1cyclictest24140-21turbostat10:20:000
27570991414,0cyclictest0-21swapper/312:20:533
27570991414,0cyclictest0-21swapper/309:40:283
27570991414,0cyclictest0-21swapper/309:30:403
2757099140,0cyclictest0-21swapper/311:10:203
2757099140,0cyclictest0-21swapper/309:00:043
2757099140,0cyclictest0-21swapper/308:10:353
27569991414,0cyclictest0-21swapper/211:06:132
27569991412,1cyclictest27304-21apt-get09:20:022
2756999140,5cyclictest0-21swapper/210:56:252
2756999140,14cyclictest0-21swapper/212:05:182
2756999140,14cyclictest0-21swapper/211:15:092
2756999140,14cyclictest0-21swapper/210:20:042
2756999140,14cyclictest0-21swapper/210:01:012
2756999140,14cyclictest0-21swapper/209:10:102
2756999140,14cyclictest0-21swapper/207:10:372
27568991414,0cyclictest0-21swapper/112:20:121
27568991414,0cyclictest0-21swapper/109:27:411
2756899140,14cyclictest0-21swapper/110:50:341
2756499146,7cyclictest611-21irqbalance10:05:360
27564991414,0cyclictest0-21swapper/009:46:230
2756499140,14cyclictest0-21swapper/010:55:010
2756499140,14cyclictest0-21swapper/010:50:030
2756499140,14cyclictest0-21swapper/009:35:150
2756499140,14cyclictest0-21swapper/008:35:140
2756499140,14cyclictest0-21swapper/007:16:360
2756499140,0cyclictest0-21swapper/012:20:040
2756499140,0cyclictest0-21swapper/011:30:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional