You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-10 - 20:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sat Jan 10, 2026 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13000993027,2cyclictest28-21ksoftirqd/210:00:002
1300099290,24cyclictest0-21swapper/209:31:392
1300099280,22cyclictest4745-21perf08:00:012
12998992622,3cyclictest606-21dbus-daemon08:45:010
1300099240,23cyclictest0-21swapper/210:26:402
1299899240,23cyclictest30843-21idleruntime-cro10:00:000
13000992321,1cyclictest6777-21awk09:10:002
1300099230,22cyclictest606-21dbus-daemon10:15:002
12999992323,0cyclictest0-21swapper/109:41:371
12999992321,1cyclictest28106-21apt-get11:01:401
12999992321,1cyclictest211rcuc/108:20:011
12999992320,3cyclictest27187-21apt-get08:46:391
1299999230,22cyclictest8065-21munin-run10:20:011
1299999230,0cyclictest0-21swapper/110:46:401
12998992321,1cyclictest21959-21perf08:35:000
12998992320,2cyclictest24964-21idleruntime-cro07:35:000
1300199220,21cyclictest7172-21apt-get09:11:393
1300199220,21cyclictest2712-21apt-get07:56:373
1300199220,21cyclictest14485-21idleruntime-cro11:40:003
1300199220,21cyclictest0-21swapper/310:46:403
1300199220,21cyclictest0-21swapper/309:06:373
1300199220,21cyclictest0-21swapper/308:41:373
1300199220,1cyclictest7133-21cron08:05:013
1300199220,1cyclictest7133-21cron08:05:013
13000992221,0cyclictest0-21swapper/211:31:382
13000992220,1cyclictest2492-21basename11:15:012
13000992220,1cyclictest20075-21munin-run10:45:012
13000992219,2cyclictest32512-21turbostat11:10:002
1300099220,21cyclictest4888-21sh11:20:012
1300099220,21cyclictest14743-21awk08:20:012
1300099220,21cyclictest0-21swapper/212:31:402
1300099220,21cyclictest0-21swapper/211:41:392
1300099220,21cyclictest0-21swapper/210:01:382
1300099220,21cyclictest0-21swapper/209:26:372
1300099220,1cyclictest24349-21perf08:40:002
12999992221,1cyclictest13242-21apt-get10:31:391
12999992220,1cyclictest0-21swapper/107:30:011
1299999221,20cyclictest8444-21apt-get10:21:401
1299999220,21cyclictest19697-21apt-get11:51:381
1299999220,21cyclictest0-21swapper/111:56:391
1299999220,21cyclictest0-21swapper/111:21:381
1299999220,21cyclictest0-21swapper/110:36:391
1299999220,1cyclictest25634-21apt-get10:56:391
1299999220,1cyclictest0-21swapper/112:01:391
12998992221,0cyclictest14383-21apt-get09:26:370
12998992220,1cyclictest9182-21grep09:15:000
12998992220,1cyclictest26536-21munin-run12:05:010
12998992220,1cyclictest12333-21perf08:15:010
12998992220,1cyclictest0-21swapper/012:00:010
12998992219,3cyclictest0-21swapper/012:31:400
1299899220,21cyclictest2380-21apt-get09:01:390
1299899220,21cyclictest14906-21apt-get11:41:390
1299899220,21cyclictest13367-21apt-get07:11:390
1299899220,21cyclictest0-21swapper/011:35:010
1299899220,21cyclictest0-21swapper/011:01:390
1299899220,21cyclictest0-21swapper/009:41:380
1299899220,21cyclictest0-21swapper/008:56:380
1299899220,21cyclictest0-21swapper/008:01:380
1299899220,21cyclictest0-21swapper/008:01:380
1299899220,21cyclictest0-21swapper/007:26:370
1299899220,1cyclictest5294-21apt-get11:21:380
1299899220,1cyclictest20198-21awk07:25:010
1299899220,1cyclictest10983-21munin-run12:40:000
13001992121,0cyclictest0-21swapper/311:46:383
13001992121,0cyclictest0-21swapper/309:46:423
13001992120,1cyclictest0-21swapper/312:16:413
13001992120,0cyclictest0-21swapper/308:26:413
13001992119,1cyclictest0-21swapper/311:30:013
13001992119,1cyclictest0-21swapper/310:21:413
1300199210,21cyclictest21200-21perf09:40:013
1300199210,21cyclictest0-21swapper/308:50:023
1300199210,20cyclictest4147-21apt-get12:26:383
1300199210,20cyclictest0-21swapper/310:51:393
1300199210,20cyclictest0-21swapper/310:36:393
1300199210,1cyclictest0-21swapper/310:41:393
13000992121,0cyclictest22971-21apt-get07:31:392
13000992120,1cyclictest0-21swapper/212:30:002
13000992120,1cyclictest0-21swapper/211:26:402
13000992120,0cyclictest20573-21apt-get07:26:372
13000992119,1cyclictest156382sleep210:36:392
13000992119,1cyclictest13367-21apt-get07:11:382
13000992119,1cyclictest12997-21cyclictest07:50:002
13000992119,1cyclictest11557-21cron09:20:002
1300099210,21cyclictest0-21swapper/208:05:022
1300099210,20cyclictest8936-21apt-get12:36:352
1300099210,20cyclictest24737-21apt-get08:41:382
1300099210,20cyclictest19925-21apt-get08:31:382
1300099210,0cyclictest0-21swapper/210:56:392
1300099210,0cyclictest0-21swapper/209:21:382
1300099210,0cyclictest0-21swapper/207:41:382
12999992121,0cyclictest0-21swapper/112:26:381
12999992118,2cyclictest32506-21sh11:10:001
1299999210,21cyclictest0-21swapper/109:26:371
1299999210,1cyclictest16784-21apt-get09:31:391
1299999210,0cyclictest0-21swapper/109:06:361
12998992121,0cyclictest0-21swapper/007:51:380
12998992119,1cyclictest1220-21apt-get10:06:390
1299899210,21cyclictest0-21swapper/010:46:400
1299899210,20cyclictest15635-21apt-get10:36:390
1299899210,20cyclictest0-21swapper/011:46:380
1299899210,0cyclictest27939-21apt-get07:41:380
13001992020,0cyclictest0-21swapper/311:11:373
13001992020,0cyclictest0-21swapper/310:31:393
13001992020,0cyclictest0-21swapper/307:51:383
13001992019,1cyclictest28966-21perf12:10:013
13001992019,1cyclictest28966-21perf12:10:013
13001992019,1cyclictest26527-21perf12:05:013
13001992019,1cyclictest15386-21perf07:15:013
13001992019,0cyclictest16784-21apt-get09:31:393
13001992017,2cyclictest30119-21latency_hist11:05:003
13001992017,2cyclictest27538-21grep07:40:003
1300199200,19cyclictest0-21swapper/308:36:383
1300199200,0cyclictest0-21swapper/308:11:393
1300199200,0cyclictest0-21swapper/307:31:393
13000992020,0cyclictest32372-21apt-get08:56:382
13000992020,0cyclictest0-21swapper/208:26:412
13000992020,0cyclictest0-21swapper/208:01:382
13000992020,0cyclictest0-21swapper/208:01:382
13000992019,1cyclictest0-21swapper/207:36:382
13000992018,1cyclictest1353-21latency_hist12:20:012
1300099200,1cyclictest21592-21apt-get09:41:382
12999992020,0cyclictest0-21swapper/111:46:381
12999992020,0cyclictest0-21swapper/111:26:391
12999992020,0cyclictest0-21swapper/110:26:401
12999992020,0cyclictest0-21swapper/108:56:381
12999992019,1cyclictest15761-21apt-get07:16:391
12999992018,1cyclictest8564-21latency_hist12:35:001
12999992018,1cyclictest4400-21latency_hist09:05:011
12998992020,0cyclictest30334-21apt-get07:46:390
12998992020,0cyclictest0-21swapper/011:26:400
12998992020,0cyclictest0-21swapper/010:11:380
12998992020,0cyclictest0-21swapper/007:16:390
12998992018,1cyclictest29354-21apt-get12:11:410
12998992018,1cyclictest26010-21perf09:50:010
12998992017,2cyclictest14504-21perf11:40:000
1299899200,19cyclictest0-21swapper/010:26:400
1299899200,0cyclictest0-21swapper/010:56:390
1299899200,0cyclictest0-21swapper/009:31:390
13001991919,0cyclictest10088-21apt-get11:31:383
13001991919,0cyclictest0-21swapper/312:21:413
13001991919,0cyclictest0-21swapper/312:11:413
13001991919,0cyclictest0-21swapper/311:51:393
13001991919,0cyclictest0-21swapper/307:26:373
13001991918,1cyclictest6545-21apt-get12:31:403
13001991918,1cyclictest3626-21apt-get10:11:373
13001991918,1cyclictest31247-21apt-get10:01:383
13001991918,0cyclictest0-21swapper/309:26:373
1300199190,0cyclictest27187-21apt-get08:46:393
13000991919,0cyclictest0-21swapper/212:06:382
13000991919,0cyclictest0-21swapper/212:06:382
13000991919,0cyclictest0-21swapper/212:01:402
13000991919,0cyclictest0-21swapper/211:56:392
13000991919,0cyclictest0-21swapper/209:11:402
1300099190,18cyclictest0-21swapper/211:46:382
12999991919,0cyclictest0-21swapper/112:16:421
12999991919,0cyclictest0-21swapper/112:06:391
12999991919,0cyclictest0-21swapper/112:06:391
12999991919,0cyclictest0-21swapper/109:11:401
12999991919,0cyclictest0-21swapper/107:46:391
12999991918,1cyclictest0-21swapper/111:31:381
12999991918,0cyclictest25367-21apt-get07:36:381
12999991917,1cyclictest20202-21perf07:25:011
12999991917,1cyclictest16932-21munin-run11:45:001
12999991917,1cyclictest14536-21awk11:40:001
12998991919,0cyclictest0-21swapper/010:51:390
12998991919,0cyclictest0-21swapper/008:05:020
12998991918,1cyclictest4768-21apt-get09:06:360
12998991917,1cyclictest15261-21perf10:35:010
1299899190,19cyclictest0-21swapper/011:11:370
13001991818,0cyclictest7920-21apt-get08:06:383
13001991818,0cyclictest0-21swapper/309:41:373
13001991818,0cyclictest0-21swapper/307:41:383
13001991817,1cyclictest2877-21apt-get11:16:403
13000991818,0cyclictest0-21swapper/211:51:382
13000991818,0cyclictest0-21swapper/209:46:422
13000991816,1cyclictest31966-21cron08:55:012
12999991818,0cyclictest0-21swapper/108:31:381
12999991818,0cyclictest0-21swapper/108:11:391
12999991817,1cyclictest23239-21apt-get10:51:391
12999991817,0cyclictest0-21swapper/110:11:381
1299999180,17cyclictest0-21swapper/107:54:151
12998991818,0cyclictest0-21swapper/008:51:400
12998991818,0cyclictest0-21swapper/008:46:390
12998991818,0cyclictest0-21swapper/008:21:380
1299899180,18cyclictest0-21swapper/008:36:380
13001991717,0cyclictest0-21swapper/308:56:383
13001991717,0cyclictest0-21swapper/307:21:383
13001991716,1cyclictest11981-21apt-get09:21:383
1300199170,16cyclictest17790-21sendmail-msp07:20:013
13000991717,0cyclictest0-21swapper/211:01:402
13000991717,0cyclictest0-21swapper/207:21:382
13000991717,0cyclictest0-21swapper/207:16:392
13000991716,1cyclictest5294-21apt-get11:21:382
13000991716,1cyclictest0-21swapper/209:01:392
13000991715,1cyclictest8044-21perf10:20:012
13000991715,1cyclictest29215-21dump-pmu-power08:50:012
12999991717,0cyclictest9566-21apt-get09:16:401
12999991717,0cyclictest0-21swapper/110:06:391
12999991717,0cyclictest0-21swapper/109:46:421
12999991717,0cyclictest0-21swapper/107:41:381
1299999170,17cyclictest0-21swapper/108:06:381
12998991717,0cyclictest0-21swapper/012:16:420
12998991717,0cyclictest0-21swapper/011:51:380
12998991717,0cyclictest0-21swapper/010:21:400
12998991717,0cyclictest0-21swapper/009:21:380
13001991616,0cyclictest0-21swapper/308:35:003
13001991614,1cyclictest15135-21apt-get08:21:383
13000991615,1cyclictest20843-21apt-get10:46:402
13000991614,1cyclictest12320-21turbostat08:15:012
13000991614,1cyclictest0-21swapper/209:51:402
1300099160,6cyclictest0-21swapper/209:36:402
12999991616,0cyclictest0-21swapper/109:21:381
12999991616,0cyclictest0-21swapper/108:41:371
12999991616,0cyclictest0-21swapper/108:21:381
12999991616,0cyclictest0-21swapper/108:05:011
12999991616,0cyclictest0-21swapper/108:05:011
12999991615,1cyclictest19177-21apt-get09:36:391
12999991614,1cyclictest31364-21apt12:15:011
1299999160,1cyclictest313-21systemd-journal12:40:001
1299999160,15cyclictest606-21dbus-daemon08:00:011
12998991616,0cyclictest0-21swapper/012:06:390
12998991616,0cyclictest0-21swapper/012:06:390
12998991616,0cyclictest0-21swapper/009:16:400
13001991515,0cyclictest22095-21apt-get11:56:393
13001991515,0cyclictest0-21swapper/311:21:383
13001991515,0cyclictest0-21swapper/310:06:393
13001991513,1cyclictest14770-21awk08:20:013
13000991515,0cyclictest1220-21apt-get10:06:392
13000991515,0cyclictest0-21swapper/210:21:402
13000991514,1cyclictest0-21swapper/211:36:392
13000991512,2cyclictest25241-21cron10:55:002
12999991515,0cyclictest0-21swapper/112:21:411
12999991515,0cyclictest0-21swapper/111:11:371
12999991514,1cyclictest0-21swapper/108:53:231
12998991515,0cyclictest0-21swapper/010:41:090
12998991515,0cyclictest0-21swapper/009:51:380
12998991515,0cyclictest0-21swapper/007:36:380
1299899150,1cyclictest0-21swapper/010:16:490
1300199140,14cyclictest0-21swapper/312:35:593
1300199140,14cyclictest0-21swapper/311:41:263
1300199140,14cyclictest0-21swapper/311:06:553
1300199140,14cyclictest0-21swapper/310:55:293
1300199140,14cyclictest0-21swapper/310:25:083
1300199140,14cyclictest0-21swapper/310:15:123
1300199140,14cyclictest0-21swapper/309:55:063
1300199140,14cyclictest0-21swapper/309:50:583
1300199140,14cyclictest0-21swapper/309:15:333
1300199140,0cyclictest0-21swapper/309:00:463
1300199140,0cyclictest0-21swapper/307:45:103
1300099140,14cyclictest0-21swapper/212:10:182
1300099140,14cyclictest0-21swapper/210:30:452
1300099140,14cyclictest0-21swapper/208:20:192
1300099140,14cyclictest0-21swapper/207:50:452
1300099140,0cyclictest0-21swapper/212:20:122
12999991414,0cyclictest0-21swapper/109:50:181
12999991414,0cyclictest0-21swapper/107:11:241
1299999140,14cyclictest0-21swapper/111:16:201
1299999140,14cyclictest0-21swapper/110:00:281
1299999140,14cyclictest0-21swapper/108:25:151
1299999140,0cyclictest0-21swapper/110:40:081
12998991414,0cyclictest0-21swapper/011:05:110
12998991414,0cyclictest0-21swapper/009:35:150
1299899140,14cyclictest0-21swapper/012:25:190
1299899140,14cyclictest0-21swapper/011:15:080
1299899140,14cyclictest0-21swapper/010:00:080
1299899140,14cyclictest0-21swapper/008:25:080
1299899140,14cyclictest0-21swapper/007:55:110
1299899140,0cyclictest0-21swapper/012:20:580
1299899140,0cyclictest0-21swapper/008:15:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional