You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 16:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Fri Jan 09, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1597199251,2cyclictest0-21swapper/009:06:330
1597499230,2cyclictest31003-21grep11:00:013
1597399230,2cyclictest313-21systemd-journal10:05:012
1597399230,23cyclictest0-21swapper/208:51:332
1597399230,22cyclictest28603-21grep10:55:012
1597399230,22cyclictest16542-21cron09:25:012
1597299230,22cyclictest18708-21apt-get07:16:331
15971992323,0cyclictest0-21swapper/011:31:340
15971992321,1cyclictest313-21systemd-journal11:05:010
15971992321,1cyclictest20653-21apt-get11:46:330
1597199230,2cyclictest25072-21turbostat.cron11:55:010
1597199230,22cyclictest7113-21latency_hist12:25:000
1597199230,22cyclictest13066-21grep11:30:000
15974992220,1cyclictest5681-21turbostat.cron07:55:013
15974992220,1cyclictest0-21swapper/312:31:333
1597499220,2cyclictest22513-21awk08:30:013
1597499220,21cyclictest6228-21apt-get11:16:333
1597499220,21cyclictest20741-21perf07:20:003
1597499220,21cyclictest0-21swapper/310:26:343
15973992220,1cyclictest7130-21apt-get12:25:002
15973992220,1cyclictest30254-21apt-get12:06:342
1597399220,21cyclictest3662-21apt-get07:51:312
1597399220,21cyclictest30463-21perf07:40:012
1597399220,21cyclictest25445-21apt-get11:56:342
1597399220,21cyclictest19384-21apt-get10:36:352
1597399220,21cyclictest13439-21apt-get11:31:332
1597399220,21cyclictest0-21swapper/209:51:332
1597399220,1cyclictest22505-21munin-run08:30:012
15972992220,2cyclictest14136-21perf09:20:011
1597299220,21cyclictest0-21swapper/111:56:341
1597299220,21cyclictest0-21swapper/110:36:351
1597299220,1cyclictest4935-21apt-get09:01:321
1597299220,1cyclictest0-21swapper/111:00:011
15971992221,1cyclictest0-21swapper/011:00:010
15971992220,1cyclictest31731-21apt-get09:56:340
15971992220,1cyclictest27484-21apt12:00:010
15971992220,1cyclictest22504-21perf08:30:010
1597199220,21cyclictest4697-21tr12:20:000
1597199220,21cyclictest27951-21munin-run07:35:010
1597199220,21cyclictest24151-21apt-get09:41:330
1597199220,21cyclictest0-21swapper/010:46:330
1597199220,21cyclictest0-21swapper/009:11:350
1597199220,21cyclictest0-21swapper/008:56:330
1597199220,21cyclictest0-21swapper/008:06:330
1597199220,21cyclictest0-21swapper/007:50:010
1597199220,1cyclictest0-21swapper/012:11:350
15974992121,0cyclictest0-21swapper/310:41:343
15974992120,1cyclictest29335-21apt-get09:51:333
15974992120,1cyclictest26183-21perf09:45:003
15974992120,1cyclictest17715-21munin-run08:20:013
15974992120,0cyclictest0-21swapper/312:26:313
15974992120,0cyclictest0-21swapper/308:46:333
15974992120,0cyclictest0-21swapper/308:06:333
15974992119,1cyclictest622-21gdbus09:40:003
15974992119,1cyclictest0-21swapper/311:56:343
15974992119,1cyclictest0-21swapper/310:20:023
15974992119,1cyclictest0-21swapper/310:00:023
1597499210,21cyclictest0-21swapper/309:46:323
1597499210,20cyclictest0-21swapper/312:01:333
1597499210,20cyclictest0-21swapper/311:11:343
1597499210,0cyclictest0-21swapper/310:46:333
1597499210,0cyclictest0-21swapper/308:56:333
15973992121,0cyclictest0-21swapper/212:26:322
15973992121,0cyclictest0-21swapper/208:41:332
15973992120,1cyclictest0-21swapper/208:46:332
15973992120,0cyclictest3836-21apt-get11:11:342
1597399210,20cyclictest16941-21apt-get09:26:332
1597399210,20cyclictest0-21swapper/210:06:322
1597399210,0cyclictest0-21swapper/208:01:342
15972992121,0cyclictest26940-21apt-get09:46:321
15972992121,0cyclictest0-21swapper/112:11:351
15972992120,1cyclictest27861-21apt-get12:01:331
15972992120,1cyclictest2190-21apt-get10:01:331
15972992120,1cyclictest12188-21apt-get10:21:331
15972992119,1cyclictest30080-21apt-get08:46:331
15972992119,1cyclictest17683-21perf08:20:011
15972992119,1cyclictest0-21swapper/108:06:331
1597299210,21cyclictest0-21swapper/108:26:331
1597299210,20cyclictest2896-21apt-get07:50:021
1597299210,20cyclictest0-21swapper/111:11:341
1597299210,20cyclictest0-21swapper/109:21:331
1597299210,0cyclictest7130-21apt-get12:25:021
1597299210,0cyclictest0-21swapper/111:26:341
1597299210,0cyclictest0-21swapper/110:46:331
1597299210,0cyclictest0-21swapper/109:56:341
1597299210,0cyclictest0-21swapper/109:26:341
1597299210,0cyclictest0-21swapper/108:56:331
1597299210,0cyclictest0-21swapper/107:21:331
15971992120,1cyclictest0-21swapper/007:51:310
15971992120,0cyclictest26166-21apt-get09:45:020
1597199210,21cyclictest0-21swapper/008:16:320
1597199210,20cyclictest12206-21apt-get12:33:490
1597199210,20cyclictest0-21swapper/012:26:320
1597199210,20cyclictest0-21swapper/010:41:340
1597199210,20cyclictest0-21swapper/010:36:350
1597199210,20cyclictest0-21swapper/007:21:330
15974992020,0cyclictest18261-21apt-get11:41:353
15974992020,0cyclictest0-21swapper/312:36:343
15974992020,0cyclictest0-21swapper/310:36:353
15974992020,0cyclictest0-21swapper/309:31:323
15974992020,0cyclictest0-21swapper/308:21:313
15974992020,0cyclictest0-21swapper/308:11:343
15974992019,1cyclictest0-21swapper/309:26:343
15974992019,1cyclictest0-21swapper/307:46:333
1597499200,0cyclictest0-21swapper/311:35:003
15973992020,0cyclictest2518-21apt-get08:56:332
15973992020,0cyclictest13265-21apt-get08:11:342
15973992020,0cyclictest10871-21apt-get08:06:332
15973992020,0cyclictest0-21swapper/209:56:342
15973992020,0cyclictest0-21swapper/209:31:322
15973992019,1cyclictest0-21swapper/209:41:322
15973992019,0cyclictest6980-21apt-get10:11:342
15973992018,1cyclictest27293-21perf08:40:012
15973992018,1cyclictest21799-21apt-get10:41:342
1597399200,20cyclictest0-21swapper/211:16:342
1597399200,20cyclictest0-21swapper/209:36:342
1597399200,20cyclictest0-21swapper/207:16:322
1597399200,19cyclictest11732-21cron09:15:012
1597399200,0cyclictest0-21swapper/209:46:322
15972992020,0cyclictest14582-21apt-get10:26:341
15972992020,0cyclictest0-21swapper/112:31:321
15972992020,0cyclictest0-21swapper/111:41:351
15972992020,0cyclictest0-21swapper/111:35:001
15972992020,0cyclictest0-21swapper/111:16:331
15972992020,0cyclictest0-21swapper/110:41:341
15972992020,0cyclictest0-21swapper/109:51:331
15972992019,1cyclictest18084-21apt-get08:21:311
15972992018,1cyclictest26166-21apt-get09:45:001
15971992020,0cyclictest16941-21apt-get09:26:340
15971992020,0cyclictest0-21swapper/011:06:340
15971992020,0cyclictest0-21swapper/010:31:340
15971992020,0cyclictest0-21swapper/009:51:330
15971992020,0cyclictest0-21swapper/009:01:320
15971992020,0cyclictest0-21swapper/007:26:320
15971992019,1cyclictest0-21swapper/011:21:320
15971992019,1cyclictest0-21swapper/011:11:340
15971992018,1cyclictest20756-21munin-run07:20:000
1597199200,20cyclictest0-21swapper/012:01:330
1597199200,0cyclictest0-21swapper/010:21:330
1597199200,0cyclictest0-21swapper/008:11:330
15974991919,0cyclictest0-21swapper/311:46:333
15974991919,0cyclictest0-21swapper/310:16:333
15974991919,0cyclictest0-21swapper/310:11:343
15974991919,0cyclictest0-21swapper/309:21:333
15974991919,0cyclictest0-21swapper/308:41:333
15974991919,0cyclictest0-21swapper/308:36:323
15974991919,0cyclictest0-21swapper/308:01:333
15974991919,0cyclictest0-21swapper/307:31:323
15974991918,0cyclictest0-21swapper/307:41:343
15973991919,0cyclictest0-21swapper/211:51:332
15973991919,0cyclictest0-21swapper/211:46:332
15973991919,0cyclictest0-21swapper/208:21:312
15973991919,0cyclictest0-21swapper/208:16:322
15973991917,1cyclictest0-21swapper/212:33:492
15972991919,0cyclictest0-21swapper/112:36:341
15972991919,0cyclictest0-21swapper/109:31:321
15972991919,0cyclictest0-21swapper/108:11:341
15972991918,1cyclictest25919-21apt-get07:31:321
15972991917,1cyclictest23751-21apt09:40:001
15971991919,0cyclictest14540-21apt-get09:21:330
15971991917,1cyclictest14171-21munin-run09:20:010
15971991916,2cyclictest13-21ksoftirqd/009:40:000
1597199190,18cyclictest0-21swapper/010:26:340
15974991818,0cyclictest0-21swapper/312:06:343
15974991818,0cyclictest0-21swapper/307:26:323
15974991816,1cyclictest606-21dbus-daemon10:10:013
15973991818,0cyclictest32650-21apt-get12:11:352
15973991818,0cyclictest0-21swapper/211:01:342
15972991818,0cyclictest0-21swapper/111:46:331
15972991818,0cyclictest0-21swapper/110:11:341
15972991818,0cyclictest0-21swapper/108:36:321
15972991818,0cyclictest0-21swapper/108:01:331
15972991817,1cyclictest0-21swapper/107:26:321
15972991817,0cyclictest30834-21apt-get07:41:341
15972991817,0cyclictest0-21swapper/110:16:331
15971991818,0cyclictest0-21swapper/011:16:330
15971991817,0cyclictest0-21swapper/011:41:350
15971991816,1cyclictest12738-21apt-get12:36:340
15974991717,0cyclictest0-21swapper/310:51:353
15974991717,0cyclictest0-21swapper/309:56:343
1597499170,9cyclictest0-21swapper/312:11:353
15973991717,0cyclictest22878-21apt-get08:31:332
15973991717,0cyclictest0-21swapper/212:36:332
15973991717,0cyclictest0-21swapper/210:31:342
15973991715,1cyclictest23140-21idleruntime-cro07:25:002
1597399171,15cyclictest15833-21apt-get11:36:332
15972991717,0cyclictest0-21swapper/112:06:341
15972991717,0cyclictest0-21swapper/108:41:331
15972991717,0cyclictest0-21swapper/107:46:331
1597299170,16cyclictest9731-21apt-get09:11:351
15971991717,0cyclictest0-21swapper/010:01:330
15971991717,0cyclictest0-21swapper/008:46:330
1597199170,16cyclictest32272-21perf12:10:000
15974991616,0cyclictest0-21swapper/312:16:353
15974991616,0cyclictest0-21swapper/311:06:343
15974991615,0cyclictest0-21swapper/310:31:343
15974991614,1cyclictest0-21swapper/311:40:013
1597499160,15cyclictest23130-21idleruntime-cro07:25:003
1597399162,12cyclictest313-21systemd-journal07:15:002
15973991616,0cyclictest0-21swapper/210:20:022
15973991616,0cyclictest0-21swapper/209:01:322
15973991615,1cyclictest11043-21apt-get11:26:342
15973991614,1cyclictest9359-21munin-run09:10:012
15973991614,1cyclictest0-21swapper/212:17:132
15972991614,1cyclictest6597-21grep10:10:011
15972991614,1cyclictest313-21systemd-journal11:05:361
15971991616,0cyclictest0-21swapper/009:31:320
15971991616,0cyclictest0-21swapper/008:21:310
15971991615,0cyclictest8477-21apt-get08:01:330
15971991615,0cyclictest0-21swapper/010:11:340
15971991615,0cyclictest0-21swapper/008:36:320
15974991515,0cyclictest0-21swapper/308:31:323
15974991515,0cyclictest0-21swapper/307:11:333
15974991514,1cyclictest0-21swapper/312:21:333
15973991515,0cyclictest801-21apt-get07:46:332
15973991515,0cyclictest0-21swapper/210:26:332
15973991514,1cyclictest12122-21apt-get09:16:332
15973991512,2cyclictest8082-21ls08:00:002
1597399150,1cyclictest31379-21tune2fs07:41:362
1597299154,8cyclictest0-21swapper/107:35:491
15972991513,1cyclictest5107-21apt-get12:21:331
15972991513,1cyclictest17915-21date11:40:011
1597299151,10cyclictest611-21irqbalance07:11:171
1597299150,1cyclictest8501-21pmu-power09:06:401
1597299150,1cyclictest32475-21apt-get08:51:331
15971991514,0cyclictest9373-21apt-get10:16:330
1597199150,14cyclictest0-21swapper/008:52:330
15974991414,0cyclictest0-21swapper/308:53:313
15974991413,1cyclictest0-21swapper/311:00:043
1597499140,14cyclictest0-21swapper/311:25:123
1597499140,14cyclictest0-21swapper/309:05:053
1597499140,14cyclictest0-21swapper/307:55:173
1597499140,14cyclictest0-21swapper/307:35:113
1597499140,0cyclictest0-21swapper/311:51:073
1597499140,0cyclictest0-21swapper/311:21:303
1597499140,0cyclictest0-21swapper/309:15:373
1597499140,0cyclictest0-21swapper/309:10:093
1597499140,0cyclictest0-21swapper/309:01:053
15973991414,0cyclictest0-21swapper/207:31:432
1597399140,14cyclictest0-21swapper/212:00:112
1597399140,14cyclictest0-21swapper/211:22:152
1597399140,14cyclictest0-21swapper/211:05:022
1597399140,14cyclictest0-21swapper/210:45:012
1597399140,14cyclictest0-21swapper/210:16:132
1597399140,14cyclictest0-21swapper/207:25:292
1597399140,0cyclictest0-21swapper/211:40:162
1597399140,0cyclictest0-21swapper/210:55:012
15972991414,0cyclictest0-21swapper/111:00:171
1597299140,14cyclictest0-21swapper/111:50:081
1597299140,14cyclictest0-21swapper/110:30:071
1597299140,14cyclictest0-21swapper/108:30:051
1597299140,14cyclictest0-21swapper/107:55:291
15971991414,0cyclictest0-21swapper/010:50:240
15971991414,0cyclictest0-21swapper/008:41:050
15971991414,0cyclictest0-21swapper/007:35:530
15971991414,0cyclictest0-21swapper/007:10:530
1597199140,14cyclictest0-21swapper/011:35:070
1597199140,14cyclictest0-21swapper/007:55:350
1597199140,14cyclictest0-21swapper/007:40:050
1597199140,0cyclictest0-21swapper/010:05:350
1597199140,0cyclictest0-21swapper/008:30:340
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional