You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 15:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sat Dec 06, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2219299250,24cyclictest12322-21idleruntime-cro12:20:000
22195992422,1cyclictest9855-21cron07:50:003
22195992321,1cyclictest27442-21cron11:45:003
2219499230,22cyclictest0-21swapper/209:23:282
22193992321,1cyclictest21944-21perf12:40:001
22193992320,2cyclictest22652-21apt11:35:001
22195992221,1cyclictest20454-21apt-get08:13:233
22195992220,1cyclictest12850-21apt-get07:58:233
22195992219,2cyclictest19953-21munin-plugin-st08:10:003
2219599220,21cyclictest7277-21apt-get08:53:243
2219599220,21cyclictest5725-21turbostat11:00:003
2219599220,21cyclictest4881-21apt-get08:48:253
2219599220,21cyclictest30539-21perf10:45:003
2219599220,21cyclictest24751-21sendmail08:20:003
2219599220,21cyclictest0-21swapper/312:33:263
2219599220,21cyclictest0-21swapper/312:08:263
22194992220,1cyclictest23165-21apt-get11:38:252
22194992220,1cyclictest0-21swapper/209:38:262
2219499220,2cyclictest14769-21dpkg08:00:012
2219499220,21cyclictest593-21dbus-daemon11:00:002
2219499220,21cyclictest461-21apt-get10:50:012
2219499220,21cyclictest19344-21apt-get09:18:262
2219499220,1cyclictest6302-21apt-get11:03:262
2219499220,1cyclictest18816-21latency_hist09:15:002
22193992221,0cyclictest0-21swapper/107:33:231
22193992220,1cyclictest29876-21perf11:49:591
22193992220,1cyclictest2219-21cron12:00:011
2219399220,21cyclictest6161-21apt-get09:58:241
2219399220,21cyclictest27175-21perf08:25:001
2219399220,21cyclictest20454-21apt-get08:13:231
2219399220,21cyclictest0-21swapper/112:28:251
2219399220,21cyclictest0-21swapper/107:58:221
2219399220,1cyclictest313-21systemd-journal08:10:001
22192992221,1cyclictest11372-21apt-get10:08:260
22192992220,1cyclictest2168-21apt-get07:35:000
22192992220,1cyclictest18793-21cron09:15:000
22192992220,1cyclictest0-21swapper/010:43:250
2219299220,21cyclictest2277-21munin-run12:00:010
2219299220,21cyclictest18365-21apt-get11:28:270
2219299220,21cyclictest0-21swapper/011:18:240
2219299220,21cyclictest0-21swapper/009:48:250
2219299220,21cyclictest0-21swapper/009:33:250
2219299220,21cyclictest0-21swapper/009:18:260
2219299220,21cyclictest0-21swapper/008:28:260
2219299220,1cyclictest24773-21grep08:20:000
22195992121,0cyclictest0-21swapper/308:03:253
22195992119,2cyclictest27679-21apt-get08:28:253
22195992119,1cyclictest30857-21turbostat09:40:013
2219599210,20cyclictest8180-21perf11:05:003
2219599210,20cyclictest31379-21apt-get09:43:253
2219599210,20cyclictest0-21swapper/312:28:253
2219599210,20cyclictest0-21swapper/311:33:263
2219599210,20cyclictest0-21swapper/311:23:263
2219599210,20cyclictest0-21swapper/311:13:263
2219599210,20cyclictest0-21swapper/310:50:003
2219599210,20cyclictest0-21swapper/310:38:253
2219599210,20cyclictest0-21swapper/309:28:253
2219599210,20cyclictest0-21swapper/309:18:263
2219599210,20cyclictest0-21swapper/309:08:243
22194992120,1cyclictest32702-21apt-get07:33:232
22194992120,0cyclictest18061-21apt-get08:08:242
22194992120,0cyclictest15964-21apt-get11:23:262
22194992119,1cyclictest22640-21wc11:35:002
2219499210,21cyclictest0-21swapper/211:28:282
2219499210,20cyclictest31971-21munin-run08:35:002
2219499210,20cyclictest27512-21apt-get07:23:232
2219499210,20cyclictest0-21swapper/212:08:252
2219499210,20cyclictest0-21swapper/210:23:242
2219499210,20cyclictest0-21swapper/209:53:232
2219499210,0cyclictest0-21swapper/211:43:242
22193992118,2cyclictest20529-21grep10:25:011
2219399210,21cyclictest0-21swapper/109:23:281
2219399210,21cyclictest0-21swapper/109:13:241
2219399210,20cyclictest8585-21apt-get10:03:291
2219399210,20cyclictest0-21swapper/108:48:251
2219399210,0cyclictest0-21swapper/110:43:251
2219399210,0cyclictest0-21swapper/109:53:221
2219399210,0cyclictest0-21swapper/108:00:011
2219399210,0cyclictest0-21swapper/107:18:231
22192992120,1cyclictest0-21swapper/011:00:320
22192992119,1cyclictest0-21swapper/011:35:000
22192992118,2cyclictest9881-21latency_hist07:50:000
2219299211,1cyclictest1-21systemd07:20:000
2219299210,2cyclictest25324-21latency_hist10:35:000
2219299210,2cyclictest16917-21head12:28:340
2219299210,20cyclictest4652-21apt-get12:05:030
2219299210,20cyclictest21764-21apt-get09:23:280
2219299210,20cyclictest0-21swapper/008:43:230
2219299210,20cyclictest0-21swapper/008:08:250
2219299210,20cyclictest0-21swapper/007:23:240
2219299210,0cyclictest0-21swapper/007:35:010
22195992020,0cyclictest4668-21perf12:05:003
22195992020,0cyclictest0-21swapper/311:58:403
22195992020,0cyclictest0-21swapper/311:28:283
22195992020,0cyclictest0-21swapper/311:08:243
22195992020,0cyclictest0-21swapper/310:33:253
22195992020,0cyclictest0-21swapper/310:13:243
22195992020,0cyclictest0-21swapper/309:13:243
22195992020,0cyclictest0-21swapper/307:53:243
22195992018,1cyclictest30077-21apt-get08:33:253
2219599200,20cyclictest0-21swapper/309:33:263
2219599200,0cyclictest0-21swapper/312:13:263
2219599200,0cyclictest0-21swapper/311:18:243
2219599200,0cyclictest0-21swapper/307:28:263
22194992020,0cyclictest22697-21apt-get07:13:262
22194992020,0cyclictest0-21swapper/212:33:272
22194992020,0cyclictest0-21swapper/212:28:252
22194992020,0cyclictest0-21swapper/212:23:242
22194992020,0cyclictest0-21swapper/211:58:402
22194992020,0cyclictest0-21swapper/209:43:252
22194992020,0cyclictest0-21swapper/208:18:232
22194992020,0cyclictest0-21swapper/207:48:232
22194992018,1cyclictest0-21swapper/212:20:002
2219499200,19cyclictest0-21swapper/209:33:262
2219499200,0cyclictest0-21swapper/212:03:252
2219499200,0cyclictest0-21swapper/208:53:242
22193992020,0cyclictest0-21swapper/110:38:241
22193992020,0cyclictest0-21swapper/107:38:251
22193992019,1cyclictest10378-21apt-get12:18:281
22193992018,1cyclictest9844-21dump-pmu-power12:15:001
22193992018,1cyclictest1974-21munin-run08:40:001
2219399200,0cyclictest0-21swapper/111:23:261
2219399200,0cyclictest0-21swapper/110:48:241
22192992020,0cyclictest25281-21apt-get08:23:250
22192992020,0cyclictest0-21swapper/010:38:250
22192992020,0cyclictest0-21swapper/009:53:230
22192992020,0cyclictest0-21swapper/009:28:250
22192992020,0cyclictest0-21swapper/008:58:250
22192992020,0cyclictest0-21swapper/008:53:240
22192992020,0cyclictest0-21swapper/008:33:250
22192992020,0cyclictest0-21swapper/008:03:250
22192992019,1cyclictest0-21swapper/008:38:220
2219299200,20cyclictest0-21swapper/009:43:250
2219299200,19cyclictest12328-21latency_hist07:55:000
2219299200,19cyclictest0-21swapper/009:03:310
2219299200,0cyclictest0-21swapper/011:48:240
2219299200,0cyclictest0-21swapper/011:23:260
2219299200,0cyclictest0-21swapper/010:18:250
22195991919,0cyclictest0-21swapper/310:53:253
22195991919,0cyclictest0-21swapper/310:28:243
22195991919,0cyclictest0-21swapper/310:08:263
22195991919,0cyclictest0-21swapper/309:03:253
22195991918,0cyclictest0-21swapper/308:23:243
22195991917,1cyclictest3764-21apt-get09:53:233
22194991919,0cyclictest24165-21apt-get09:28:262
22194991919,0cyclictest0-21swapper/211:53:252
22194991919,0cyclictest0-21swapper/211:13:272
22194991919,0cyclictest0-21swapper/210:18:242
22194991919,0cyclictest0-21swapper/208:38:232
22194991919,0cyclictest0-21swapper/207:43:242
22194991918,1cyclictest1354-21apt-get09:48:252
22193991919,0cyclictest13544-21apt-get11:18:241
22193991919,0cyclictest0-21swapper/109:18:261
22193991919,0cyclictest0-21swapper/108:43:231
22193991919,0cyclictest0-21swapper/107:28:261
22193991919,0cyclictest0-21swapper/107:23:241
22193991917,1cyclictest13303-21latency_hist10:10:001
22193991917,1cyclictest12146-21apt-get09:03:251
22193991917,1cyclictest10400-21apt-get07:53:241
22193991916,2cyclictest31967-21perf08:35:001
2219399190,15cyclictest273132sleep109:33:291
22192991919,0cyclictest0-21swapper/012:23:240
22192991919,0cyclictest0-21swapper/011:38:250
22192991919,0cyclictest0-21swapper/010:23:240
22192991919,0cyclictest0-21swapper/008:13:240
22192991919,0cyclictest0-21swapper/007:43:240
22192991919,0cyclictest0-21swapper/007:13:250
22192991914,4cyclictest14750-21latency_hist08:00:010
22195991818,0cyclictest18639-21apt-get10:23:243
22195991818,0cyclictest0-21swapper/312:38:263
22195991818,0cyclictest0-21swapper/308:38:233
22195991818,0cyclictest0-21swapper/307:33:233
22194991818,0cyclictest0-21swapper/210:03:282
22194991817,0cyclictest0-21swapper/208:13:232
22193991818,0cyclictest21034-21apt-get10:28:241
22193991818,0cyclictest0-21swapper/110:18:251
22193991818,0cyclictest0-21swapper/110:13:251
22193991818,0cyclictest0-21swapper/109:08:241
22193991816,1cyclictest8697-21apt-get11:08:251
22193991816,1cyclictest25584-21apt-get11:43:241
22193991816,1cyclictest24589-21munin-run07:15:011
2219399180,0cyclictest0-21swapper/110:33:251
22192991818,0cyclictest0-21swapper/012:38:260
22192991818,0cyclictest0-21swapper/011:53:250
22192991818,0cyclictest0-21swapper/009:08:240
22192991817,1cyclictest17641-21apt-get12:33:270
22192991817,1cyclictest0-21swapper/012:13:250
22195991717,0cyclictest0-21swapper/311:53:253
22195991717,0cyclictest0-21swapper/307:43:243
22194991717,0cyclictest0-21swapper/208:58:252
22194991716,1cyclictest15665-21apt-get08:03:252
22194991716,1cyclictest14039-21perf09:05:002
22194991716,1cyclictest0-21swapper/211:08:252
22194991715,1cyclictest27174-21munin-run08:25:002
22193991717,0cyclictest9674-21apt-get08:58:251
22193991717,0cyclictest0-21swapper/111:03:261
22193991717,0cyclictest0-21swapper/109:38:251
22192991717,0cyclictest0-21swapper/011:43:240
22195991616,0cyclictest0-21swapper/309:48:253
22195991614,1cyclictest26989-21munin-run07:20:003
22194991615,1cyclictest13821-21apt-get10:13:252
22194991615,1cyclictest0-21swapper/211:48:242
22194991614,1cyclictest0-21swapper/208:48:352
2219499160,1cyclictest0-21swapper/207:53:242
2219499160,15cyclictest330-21taskset10:48:432
2219499160,15cyclictest0-21swapper/211:20:002
22193991616,0cyclictest0-21swapper/111:53:251
22193991616,0cyclictest0-21swapper/109:48:251
2219399160,15cyclictest2785-21apt-get12:03:251
2219399160,15cyclictest2704-21cut10:53:321
22192991616,0cyclictest0-21swapper/010:48:240
22192991616,0cyclictest0-21swapper/010:13:250
22192991614,1cyclictest31796-21munin-run07:30:000
22192991614,1cyclictest11093-21apt-get11:13:260
22192991614,1cyclictest10590-21munin-run11:10:010
2219299160,15cyclictest0-21swapper/010:53:270
22195991515,0cyclictest0-21swapper/310:03:293
22195991515,0cyclictest0-21swapper/307:13:263
22195991514,1cyclictest0-21swapper/308:58:263
22194991515,0cyclictest28653-21apt-get10:43:252
22194991515,0cyclictest27010-21sendmail-msp07:20:002
22194991515,0cyclictest0-21swapper/209:58:242
22194991513,1cyclictest0-21swapper/210:28:312
2219499150,14cyclictest0-21swapper/208:25:152
2219499150,0cyclictest0-21swapper/210:36:582
22193991515,0cyclictest0-21swapper/111:13:271
22193991513,1cyclictest8326-21ntp_states08:53:291
22192991513,1cyclictest0-21swapper/012:03:270
2219299150,14cyclictest287842sleep009:36:320
22195991414,0cyclictest0-21swapper/307:21:333
22195991412,1cyclictest6470-21expr09:58:253
2219599140,14cyclictest0-21swapper/312:15:463
2219599140,14cyclictest0-21swapper/311:35:243
2219599140,14cyclictest0-21swapper/310:15:143
2219599140,14cyclictest0-21swapper/309:21:503
2219599140,14cyclictest0-21swapper/308:40:293
2219599140,0cyclictest0-21swapper/312:20:243
2219599140,0cyclictest0-21swapper/311:47:123
22194991414,0cyclictest0-21swapper/210:05:022
22194991414,0cyclictest0-21swapper/209:05:222
2219499140,14cyclictest0-21swapper/210:30:022
2219499140,14cyclictest0-21swapper/207:25:292
2219499140,0cyclictest0-21swapper/212:37:032
2219499140,0cyclictest0-21swapper/208:44:302
2219499140,0cyclictest0-21swapper/207:38:412
22193991414,0cyclictest0-21swapper/111:25:231
22193991414,0cyclictest0-21swapper/110:56:281
22193991414,0cyclictest0-21swapper/108:27:261
2219399140,14cyclictest0-21swapper/112:30:491
2219399140,14cyclictest0-21swapper/111:35:021
2219399140,14cyclictest0-21swapper/109:40:061
2219399140,14cyclictest0-21swapper/109:25:021
2219399140,0cyclictest0-21swapper/108:16:091
2219399140,0cyclictest0-21swapper/107:40:131
22192991414,0cyclictest0-21swapper/010:25:590
22192991414,0cyclictest0-21swapper/008:46:530
2219299140,14cyclictest0-21swapper/010:55:010
2219299140,0cyclictest0-21swapper/010:00:190
2219299140,0cyclictest0-21swapper/009:57:540
22195991313,0cyclictest0-21swapper/307:36:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional